Jul 24 14:31:58.512753 ip-10-0-141-133 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 14:31:58.512760 ip-10-0-141-133 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 14:31:58.512769 ip-10-0-141-133 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 14:31:58.513053 ip-10-0-141-133 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 14:32:07.688945 ip-10-0-141-133 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 14:32:07.688960 ip-10-0-141-133 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d4e51c11dd534ce2aa41424909602120 --
Jul 24 14:33:34.716711 ip-10-0-141-133 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.169216    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174077    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174087    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174090    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174093    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174096    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174098    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174101    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:33:35.225602 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174104    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174107    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174110    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174113    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174115    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174118    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174120    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174123    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174126    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174128    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174131    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174133    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174137    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174139    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174142    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174144    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174147    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174149    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174152    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174155    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:33:35.240091 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174157    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174160    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174162    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174165    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174167    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174170    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174172    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174175    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174177    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174180    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174182    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174185    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174187    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174189    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174192    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174194    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174197    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174200    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174205    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:33:35.242394 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174209    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174213    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174216    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174220    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174222    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174225    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174227    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174230    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174232    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174235    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174237    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174240    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174243    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174245    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174249    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174252    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174256    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174258    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174261    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:33:35.244180 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174265    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174267    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174270    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174273    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174275    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174278    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174281    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174283    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174285    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174288    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174290    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174293    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174295    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174298    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174300    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174302    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174305    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174307    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174310    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174312    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:33:35.247466 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174314    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:33:35.244931 ip-10-0-141-133 systemd[1]: Started Kubernetes Kubelet.
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174317    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174319    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174322    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174325    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174328    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174330    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174332    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174335    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174337    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174340    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174342    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174345    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.174348    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175023    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175029    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175039    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175042    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175045    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175048    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:33:35.248745 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175051    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175054    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175056    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175059    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175061    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175063    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175067    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175069    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175071    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175074    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175076    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175079    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175081    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175083    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175086    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175094    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175096    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175099    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175101    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175104    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:33:35.264625 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175106    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175108    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175111    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175114    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175116    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175119    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175121    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175124    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175126    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175129    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175131    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175134    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175137    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175140    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175142    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175145    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175147    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175149    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175152    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175154    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:33:35.266615 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175156    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175159    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175161    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175164    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175166    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175168    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175171    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175174    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175177    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175179    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175182    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175184    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175187    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175189    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175191    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175194    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175198    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175201    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175204    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:33:35.269876 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175207    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175211    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175214    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175218    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175221    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175224    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175227    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175230    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175232    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175235    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175238    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175240    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175242    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175245    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175247    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175250    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175252    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175255    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175258    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:33:35.284226 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175260    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175265    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175267    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175269    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175272    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175274    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175277    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175279    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175282    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175284    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175287    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175289    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175292    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175294    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175297    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175641    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175646    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175648    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175651    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175654    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:33:35.286032 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175657    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175659    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175662    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175665    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175667    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175670    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175672    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175675    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175677    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175680    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175682    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175684    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175687    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175691    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175694    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175696    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175699    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175701    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175704    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:33:35.438061 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175706    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175709    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175711    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175713    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175716    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175719    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175721    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175724    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175726    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175729    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175731    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175734    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175736    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175740    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175742    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175744    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175747    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175750    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175752    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175755    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:33:35.456206 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175757    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175761    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175764    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175766    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175769    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175771    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175775    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175777    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175780    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175782    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175785    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175787    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175790    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175792    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175794    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175797    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175799    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175802    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175804    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175806    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:33:35.457085 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175809    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175811    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175813    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175816    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175819    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175821    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175824    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175826    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175829    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175831    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175833    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175838    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175841    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175845    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175848    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175851    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175853    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175856    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175861    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:33:35.457929 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175864    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175866    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175869    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175871    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175874    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175876    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175879    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175881    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175884    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175886    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175888    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175891    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175893    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175896    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175898    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.175902    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.175964    2583 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.175971    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.175976    2583 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.175982    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.175989    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 14:33:35.458734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.175994    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176000    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176004    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176007    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176010    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176015    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176018    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176021    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176024    2583 flags.go:64] FLAG: --cgroup-root=""
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176027    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176031    2583 flags.go:64] FLAG: --client-ca-file=""
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176034    2583 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176037    2583 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176041    2583 flags.go:64] FLAG: --cluster-domain=""
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176044    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176046    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176050    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176053    2583 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176057    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176060    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176063    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176066    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176069    2583 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176072    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 14:33:35.459777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176074    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176077    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176080    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176086    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176089    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176092    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176095    2583 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176097    2583 flags.go:64] FLAG: --enable-server="true"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176103    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176107    2583 flags.go:64] FLAG: --event-burst="100"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176110    2583 flags.go:64] FLAG: --event-qps="50"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176112    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176115    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176119    2583 flags.go:64] FLAG: --eviction-hard=""
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176123    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176125    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176128    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176131    2583 flags.go:64] FLAG: --eviction-soft=""
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176135    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176137    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176140    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176143    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176145    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176148    2583 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176151    2583 flags.go:64] FLAG: --feature-gates=""
Jul 24 14:33:35.461752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176154    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176157    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176160    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176163    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176166    2583 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176168    2583 flags.go:64] FLAG: --help="false"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176171    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176174    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176177    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176179    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176182    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176199    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176203    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176206    2583 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176209    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176211    2583 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176215    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176218    2583 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176221    2583 flags.go:64] FLAG: --kube-reserved=""
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176224    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176228    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176231    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176233    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176236    2583 flags.go:64] FLAG: --lock-file=""
Jul 24 14:33:35.463435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176239    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176242    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176245    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176251    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176253    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176256    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176258    2583 flags.go:64] FLAG: --logging-format="text"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176261    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176264    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176267    2583 flags.go:64] FLAG: --manifest-url=""
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176270    2583 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176274    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176276    2583 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176280    2583 flags.go:64] FLAG: --max-pods="110"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176283    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176286    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176289    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176291    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176294    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176297    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176300    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176306    2583 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176308    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176311    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 14:33:35.471705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176314    2583 flags.go:64] FLAG: --pod-cidr=""
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176317    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176322    2583 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176325    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176329    2583 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176332    2583 flags.go:64] FLAG: --port="10250"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176334    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176337    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-062a18804332d88fb"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176340    2583 flags.go:64] FLAG: --qos-reserved=""
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176343    2583 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176347    2583 flags.go:64] FLAG: --register-node="true"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176350    2583 flags.go:64] FLAG: --register-with-taints=""
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176354    2583 flags.go:64] FLAG: --registry-burst="10"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176356    2583 flags.go:64] FLAG: --registry-qps="5"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176359    2583 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176362    2583 flags.go:64] FLAG: --reserved-memory=""
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176365    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176368    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176371    2583 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176374    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176377    2583 flags.go:64] FLAG: --runonce="false"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176380    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176382    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176385    2583 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176388    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 14:33:35.473589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176391    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176394    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176396    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176399    2583 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176402    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176405    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176407    2583 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176410    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176413    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176416    2583 flags.go:64] FLAG: --system-cgroups=""
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176418    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176425    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176427    2583 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176430    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176434    2583 flags.go:64] FLAG: --tls-min-version=""
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176437    2583 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176439    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176443    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176446    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176448    2583 flags.go:64] FLAG: --v="2"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176468    2583 flags.go:64] FLAG: --version="false"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176472    2583 flags.go:64] FLAG: --vmodule=""
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176476    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.176479    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176583    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:33:35.480886 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176587    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176591    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176593    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176596    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176599    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176602    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176605    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176608    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176611    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176613    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176616    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176618    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176622    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176626    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176629    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176632    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176635    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176639    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176644    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176646    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:33:35.481852 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176649    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176651    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176654    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176657    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176661    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176663    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176666    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176669    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176671    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176674    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176676    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176678    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176681    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176684    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176686    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176688    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176691    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176693    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176696    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176698    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:33:35.482397 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176700    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176703    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176705    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176707    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176710    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176712    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176715    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176718    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176720    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176723    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176727    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176729    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176732    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176734    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176737    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176739    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176743    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176745    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176748    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:33:35.482972 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176751    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176753    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176756    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176758    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176761    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176763    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176766    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176768    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176770    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176773    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176776    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176779    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176781    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176783    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176786    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176788    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176791    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176793    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176795    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176798    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:33:35.483553 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176800    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176802    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176805    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176809    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176811    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176814    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176816    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176819    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176821    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176825    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176827    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176830    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176832    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176835    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176837    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176839    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176843    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176846    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:33:35.484102 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.176849    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.177541    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.184862    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.184875    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.184893    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.184898    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184945    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184949    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184953    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184956    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184958    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184961    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184963    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184966    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184969    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:33:35.484597 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184971    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184974    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184976    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184979    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184982    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184984    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184987    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184989    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184992    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184995    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184997    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.184999    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185002    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185004    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185007    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185009    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185012    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185014    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185017    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:33:35.485033 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185020    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185022    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185025    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185027    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185030    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185033    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185037    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185041    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185043    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185046    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185049    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185051    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185054    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185056    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185059    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185061    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185064    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185068    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185070    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185073    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:33:35.485543 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185075    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185078    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185080    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185085    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185088    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185091    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185093    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185096    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185098    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185101    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185103    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185106    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185108    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185111    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185113    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185116    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185119    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185121    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185124    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185126    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:33:35.486084 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185129    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185131    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185133    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185136    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185138    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185141    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185143    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185146    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185151    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185153    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185156    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185158    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185161    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185163    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185166    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185170    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185172    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185174    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185177    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:33:35.630060 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185180    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185183    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185186    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185190    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185194    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185197    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185200    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185203    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185205    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185208    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185210    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185213    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.185217    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185334    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:33:35.630842 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185339    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185343    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185347    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185349    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185352    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185355    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185357    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185360    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185365    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185367    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185371    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185373    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185376    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185380    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185384    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185387    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185391    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185394    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185396    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:33:35.631470 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185399    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185401    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185404    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185407    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185409    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185413    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185416    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185419    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185422    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185424    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185427    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185430    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185432    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185435    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185437    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185440    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185443    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185445    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185447    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185467    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:33:35.632056 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185471    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185475    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185478    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185480    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185486    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185489    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185491    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185495    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185498    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185501    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185503    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185506    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185508    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185511    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185513    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185516    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185518    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185521    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185523    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:33:35.632584 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185526    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185528    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185531    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185533    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185536    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185538    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185541    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185543    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185545    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185548    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185550    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185553    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185555    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185558    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185561    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185564    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185566    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185568    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185571    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185574    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:33:35.633141 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185578    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185580    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185583    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185585    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185587    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185590    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185592    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185595    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185597    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185600    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185602    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185604    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185607    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185609    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185612    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185615    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185617    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185620    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185622    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:33:35.633665 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:35.185624    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.185629    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.185726    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.188054    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.189688    2583 server.go:1019] "Starting client certificate rotation"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.189785    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.189818    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.192367    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.196907    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.209384    2583 log.go:25] "Validated CRI v1 runtime API"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.214170    2583 log.go:25] "Validated CRI v1 image API"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.214938    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 14:33:35.634189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.217338    2583 fs.go:135] Filesystem UUIDs: map[2256257c-3cf3-48ee-beef-5e6f52a9255e:/dev/nvme0n1p4 2b6fadd6-8ed0-4a96-8e96-224dedc01295:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 24 14:33:35.634528 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.217356    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 14:33:35.634620 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.222570    2583 manager.go:216] Machine: {Timestamp:2026-07-24 14:33:35.221155466 +0000 UTC m=+0.374156183 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3208326 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b3c459c8ce6f94c267ccba85f143e SystemUUID:ec2b3c45-9c8c-e6f9-4c26-7ccba85f143e BootID:d4e51c11-dd53-4ce2-aa41-424909602120 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f3:50:95:7a:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f3:50:95:7a:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:29:7e:aa:51:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 14:33:35.634620 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.222656    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 14:33:35.634620 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.222713    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 14:33:35.634620 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.226135    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 14:33:35.634620 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.226161    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-133.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 14:33:35.634620 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.227046    2583 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 14:33:35.634620 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.227056    2583 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 14:33:35.634620 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.227063    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.227075    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.227788    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.228293    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.231087    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.231183    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.231209    2583 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.231217    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.231234    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.231245    2583 kubelet.go:398] "Adding apiserver pod source"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.231256    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.231524    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.231537    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.233750    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.237340    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.237372    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240333    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240348    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240354    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240359    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240367    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240372    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240380    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240385    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240390    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240396    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240404    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.240409    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.241281    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.241291    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.241810    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.241810    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.243716    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-133.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.244294    2583 server.go:1268] "Started kubelet"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.244363    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.244388    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.244472    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.246388    2583 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.246749    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 14:33:35.635165 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.251375    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.252732    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-133.ec2.internal.18c5406198f1fc22  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-133.ec2.internal,UID:ip-10-0-141-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-133.ec2.internal,},FirstTimestamp:2026-07-24 14:33:35.244270626 +0000 UTC m=+0.397271342,LastTimestamp:2026-07-24 14:33:35.244270626 +0000 UTC m=+0.397271342,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-133.ec2.internal,}"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.257628    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.257638    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.257722    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258345    2583 factory.go:55] Registering systemd factory
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258357    2583 factory.go:223] Registration of the systemd container factory successfully
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258436    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258449    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258435    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258515    2583 factory.go:153] Registering CRI-O factory
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258526    2583 factory.go:223] Registration of the crio container factory successfully
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258538    2583 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258547    2583 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258581    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258605    2583 factory.go:103] Registering Raw factory
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.258619    2583 manager.go:1196] Started watching for new ooms in manager
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.259042    2583 manager.go:349] Starting recovery of all containers
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.260109    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.260206    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.272197    2583 manager.go:354] Recovery completed
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.277524    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.279144    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.279174    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.279183    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.279653    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.279663    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.279677    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.279655    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.280692    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wq2vs"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.281145    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.281167    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.281177    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:33:35.636179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.282322    2583 policy_none.go:49] "None policy: Start"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.282334    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.282343    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.284306    2583 policy_none.go:47] "Start"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.289743    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-133.ec2.internal.18c540619b066170  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-133.ec2.internal,UID:ip-10-0-141-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-133.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-133.ec2.internal,},FirstTimestamp:2026-07-24 14:33:35.279161712 +0000 UTC m=+0.432162430,LastTimestamp:2026-07-24 14:33:35.279161712 +0000 UTC m=+0.432162430,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-133.ec2.internal,}"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.294026    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wq2vs"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.301424    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-133.ec2.internal.18c540619b06a189  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-133.ec2.internal,UID:ip-10-0-141-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-133.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-133.ec2.internal,},FirstTimestamp:2026-07-24 14:33:35.279178121 +0000 UTC m=+0.432178838,LastTimestamp:2026-07-24 14:33:35.279178121 +0000 UTC m=+0.432178838,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-133.ec2.internal,}"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.317078    2583 manager.go:337] "Starting Device Plugin manager"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.317106    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.317123    2583 server.go:87] "Starting device plugin registration server"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.317325    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.317335    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.317449    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.317536    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.317543    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.317960    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.318006    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.335180    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.336193    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.336236    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.336261    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.336301    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.338153    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.417680    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.422600    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.422624    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.422638    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:33:35.796768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.422665    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.431770    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.431786    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-133.ec2.internal\": node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.436710    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal"]
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.436772    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.438711    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.438736    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.438749    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.440918    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.441032    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.441057    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.441584    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.441602    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.441616    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.441637    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.441651    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.441660    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.443864    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.443882    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.444487    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.444505    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.444515    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.448022    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.459645    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe1e8f4bf149517fec76eac6337ba030-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"fe1e8f4bf149517fec76eac6337ba030\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.459674    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe1e8f4bf149517fec76eac6337ba030-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"fe1e8f4bf149517fec76eac6337ba030\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.459696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1557375bdf94e26d9bcd144776d91190-config\") pod \"kube-apiserver-proxy-ip-10-0-141-133.ec2.internal\" (UID: \"1557375bdf94e26d9bcd144776d91190\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.490925    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-133.ec2.internal\" not found" node="ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.797611 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.500338    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-133.ec2.internal\" not found" node="ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.548771    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.560097    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe1e8f4bf149517fec76eac6337ba030-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"fe1e8f4bf149517fec76eac6337ba030\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.560119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe1e8f4bf149517fec76eac6337ba030-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"fe1e8f4bf149517fec76eac6337ba030\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.560133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1557375bdf94e26d9bcd144776d91190-config\") pod \"kube-apiserver-proxy-ip-10-0-141-133.ec2.internal\" (UID: \"1557375bdf94e26d9bcd144776d91190\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.560159    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1557375bdf94e26d9bcd144776d91190-config\") pod \"kube-apiserver-proxy-ip-10-0-141-133.ec2.internal\" (UID: \"1557375bdf94e26d9bcd144776d91190\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.560183    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe1e8f4bf149517fec76eac6337ba030-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"fe1e8f4bf149517fec76eac6337ba030\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.560193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe1e8f4bf149517fec76eac6337ba030-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"fe1e8f4bf149517fec76eac6337ba030\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.649351    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.750068    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:35.798749 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.793518    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.915120 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:35.804382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:35.915120 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.850833    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:36.060215 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:35.951358    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:36.060215 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:36.051796    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:36.219248 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:36.152523    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:36.219248 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.188941    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 14:33:36.219248 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.189043    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 14:33:36.356238 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:36.253172    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found"
Jul 24 14:33:36.356238 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.260584    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:33:36.356238 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.300687    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 14:28:35 +0000 UTC" deadline="2028-04-26 05:02:54.092753962 +0000 UTC"
Jul 24 14:33:36.356238 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.300711    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15398h29m17.792045522s"
Jul 24 14:33:36.358132 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.357839    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:36.486486 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.360283    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:33:36.486486 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.367737    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 14:33:36.486486 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.369711    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal"
Jul 24 14:33:36.486486 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.380787    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 14:33:36.486486 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:36.477684    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1557375bdf94e26d9bcd144776d91190.slice/crio-7c23f6781edc95dc90b3a7a0a4ef860cf79a54f0b5290e31f18e2a72b9b2b05f WatchSource:0}: Error finding container 7c23f6781edc95dc90b3a7a0a4ef860cf79a54f0b5290e31f18e2a72b9b2b05f: Status 404 returned error can't find the container with id 7c23f6781edc95dc90b3a7a0a4ef860cf79a54f0b5290e31f18e2a72b9b2b05f
Jul 24 14:33:36.486486 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:36.483664    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe1e8f4bf149517fec76eac6337ba030.slice/crio-ce5afd4789b85a2d8b4c5fe604521a7369e61612db8fe9a3e5c618e8f3c33837 WatchSource:0}: Error finding container ce5afd4789b85a2d8b4c5fe604521a7369e61612db8fe9a3e5c618e8f3c33837: Status 404 returned error can't find the container with id ce5afd4789b85a2d8b4c5fe604521a7369e61612db8fe9a3e5c618e8f3c33837
Jul 24 14:33:36.486486 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.484263    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:33:36.633137 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:36.578180    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:33:37.248057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.232686    2583 apiserver.go:52] "Watching apiserver"
Jul 24 14:33:37.248057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.240716    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 14:33:37.248057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.241158    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-b6n7q","openshift-dns/node-resolver-cxg4g","openshift-image-registry/node-ca-cr2nk","openshift-multus/multus-d4n8r","openshift-multus/network-metrics-daemon-qdmlm","kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal","openshift-multus/multus-additional-cni-plugins-9rw82","openshift-network-diagnostics/network-check-target-vghl9","openshift-network-operator/iptables-alerter-tw45g","openshift-ovn-kubernetes/ovnkube-node-vqqjv","kube-system/global-pull-secret-syncer-lntn6","kube-system/konnectivity-agent-dr2g4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"]
Jul 24 14:33:37.248057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.243759    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.248057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.245965    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:37.248057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.246456    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 14:33:37.248057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.246482    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:33:37.248057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.246529    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6h779\""
Jul 24 14:33:37.248057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.246568    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 14:33:37.249427 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.248183    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 14:33:37.249427 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.248465    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-js87p\""
Jul 24 14:33:37.249427 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.248785    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 14:33:37.249427 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.248802    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.251260 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.250677    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6hlzv\""
Jul 24 14:33:37.251260 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.250878    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 14:33:37.251260 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.251025    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.255178 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.251650    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 14:33:37.255178 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.251677    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 14:33:37.255178 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.252920    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 14:33:37.255178 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.253190    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:37.255178 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.253239    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 14:33:37.255178 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.253275    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n9cjb\""
Jul 24 14:33:37.255178 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.253269    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:37.255178 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.253542    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 14:33:37.255178 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.253562    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 14:33:37.260997 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.256938    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.260997 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.258470    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 14:33:37.260997 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.259042    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 14:33:37.260997 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.259212    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 14:33:37.260997 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.259546    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zhgg5\""
Jul 24 14:33:37.260997 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.259565    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 14:33:37.271766 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.270384    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276309    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.276383    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276389    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbc4t\" (UniqueName: \"kubernetes.io/projected/28c0761e-a2fb-45db-9e96-a0464664e358-kube-api-access-jbc4t\") pod \"node-ca-cr2nk\" (UID: \"28c0761e-a2fb-45db-9e96-a0464664e358\") " pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276443    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnjr7\" (UniqueName: \"kubernetes.io/projected/10ba62ec-7006-4a09-901f-2d09df92881e-kube-api-access-tnjr7\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276491    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-sys-fs\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-registration-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276572    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4bb9cb9e-5076-44dd-8588-290beecbc993-agent-certs\") pod \"konnectivity-agent-dr2g4\" (UID: \"4bb9cb9e-5076-44dd-8588-290beecbc993\") " pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276597    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c1d1eff-c83c-4e12-b106-2181f688bb58-cni-binary-copy\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276624    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-daemon-config\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276649    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-run-netns\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276674    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-cnibin\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276699    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-run-k8s-cni-cncf-io\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.338183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276739    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-var-lib-cni-bin\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-etc-kubernetes\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276792    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-run-multus-certs\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276815    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whnzt\" (UniqueName: \"kubernetes.io/projected/8c1d1eff-c83c-4e12-b106-2181f688bb58-kube-api-access-whnzt\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276877    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbc7d\" (UniqueName: \"kubernetes.io/projected/a3c35b40-12db-4440-ac22-5146939e3455-kube-api-access-cbc7d\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28c0761e-a2fb-45db-9e96-a0464664e358-serviceca\") pod \"node-ca-cr2nk\" (UID: \"28c0761e-a2fb-45db-9e96-a0464664e358\") " pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.276970    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8830e241-ee58-449f-9066-0fdc86670723-iptables-alerter-script\") pod \"iptables-alerter-tw45g\" (UID: \"8830e241-ee58-449f-9066-0fdc86670723\") " pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277000    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-system-cni-dir\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277025    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-cni-dir\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277049    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-os-release\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277074    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-conf-dir\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277140    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277180    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-etc-selinux\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.341327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277213    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8830e241-ee58-449f-9066-0fdc86670723-host-slash\") pod \"iptables-alerter-tw45g\" (UID: \"8830e241-ee58-449f-9066-0fdc86670723\") " pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277242    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4bb9cb9e-5076-44dd-8588-290beecbc993-konnectivity-ca\") pod \"konnectivity-agent-dr2g4\" (UID: \"4bb9cb9e-5076-44dd-8588-290beecbc993\") " pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277291    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-var-lib-kubelet\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277320    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c0761e-a2fb-45db-9e96-a0464664e358-host\") pod \"node-ca-cr2nk\" (UID: \"28c0761e-a2fb-45db-9e96-a0464664e358\") " pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277345    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-socket-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277371    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-device-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277412    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-socket-dir-parent\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277475    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6xtx\" (UniqueName: \"kubernetes.io/projected/8830e241-ee58-449f-9066-0fdc86670723-kube-api-access-c6xtx\") pod \"iptables-alerter-tw45g\" (UID: \"8830e241-ee58-449f-9066-0fdc86670723\") " pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277506    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-var-lib-cni-multus\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.277532    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-hostroot\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.278992    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.281446    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.283707    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.286113    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.286182    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.286308    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.286500    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r4r4m\""
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.287212    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-czmlz\""
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.287350    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.287446    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 14:33:37.342465 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.288349    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t7zn6\""
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.287544    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.289292    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.291398    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.291753    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.291770    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.291799    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fr55d\""
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.292101    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.292156    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.292779    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.292912    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.293050    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.302377    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rtk85"
Jul 24 14:33:37.345220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.312355    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rtk85"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.349476    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" event={"ID":"fe1e8f4bf149517fec76eac6337ba030","Type":"ContainerStarted","Data":"ce5afd4789b85a2d8b4c5fe604521a7369e61612db8fe9a3e5c618e8f3c33837"}
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.351501    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" event={"ID":"1557375bdf94e26d9bcd144776d91190","Type":"ContainerStarted","Data":"7c23f6781edc95dc90b3a7a0a4ef860cf79a54f0b5290e31f18e2a72b9b2b05f"}
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.360047    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.377836    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-sys\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.377866    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-lib-modules\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.377893    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.377918    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-ovnkube-config\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.377942    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfsgk\" (UniqueName: \"kubernetes.io/projected/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-kube-api-access-pfsgk\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.377971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28c0761e-a2fb-45db-9e96-a0464664e358-serviceca\") pod \"node-ca-cr2nk\" (UID: \"28c0761e-a2fb-45db-9e96-a0464664e358\") " pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.377995    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-cni-dir\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-os-release\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378036    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-etc-selinux\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378074    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-var-lib-kubelet\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.406672 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378088    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c0761e-a2fb-45db-9e96-a0464664e358-host\") pod \"node-ca-cr2nk\" (UID: \"28c0761e-a2fb-45db-9e96-a0464664e358\") " pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378104    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-socket-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378120    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d8c42ee6-258a-4618-93b7-d8781893e27f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378144    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d8c42ee6-258a-4618-93b7-d8781893e27f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378171    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxk89\" (UniqueName: \"kubernetes.io/projected/d8c42ee6-258a-4618-93b7-d8781893e27f-kube-api-access-qxk89\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378209    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-run-openvswitch\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378240    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-hostroot\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnjr7\" (UniqueName: \"kubernetes.io/projected/10ba62ec-7006-4a09-901f-2d09df92881e-kube-api-access-tnjr7\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378292    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-var-lib-kubelet\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378313    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/585bbc0a-c23e-4e78-b089-9e41147e92c9-dbus\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378336    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-systemd-units\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378360    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-run-ovn\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378382    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-log-socket\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-kubernetes\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.407995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378429    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-run\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378457    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-ovn-node-metrics-cert\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c1d1eff-c83c-4e12-b106-2181f688bb58-cni-binary-copy\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378534    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-os-release\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378567    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7429f1ce-5023-470a-9736-077b1ab02ff4-hosts-file\") pod \"node-resolver-cxg4g\" (UID: \"7429f1ce-5023-470a-9736-077b1ab02ff4\") " pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378589    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-host\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-tuned\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-run-netns\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d8c42ee6-258a-4618-93b7-d8781893e27f-cni-binary-copy\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378691    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-etc-openvswitch\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-cnibin\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378740    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-run-k8s-cni-cncf-io\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378763    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-etc-kubernetes\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378788    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/585bbc0a-c23e-4e78-b089-9e41147e92c9-kubelet-config\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.410026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378813    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-var-lib-openvswitch\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378845    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-cni-bin\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378870    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-env-overrides\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378903    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-run-multus-certs\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.378984    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbc7d\" (UniqueName: \"kubernetes.io/projected/a3c35b40-12db-4440-ac22-5146939e3455-kube-api-access-cbc7d\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379010    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-system-cni-dir\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379036    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-sysconfig\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379060    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xndgm\" (UniqueName: \"kubernetes.io/projected/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-kube-api-access-xndgm\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379084    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-node-log\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8830e241-ee58-449f-9066-0fdc86670723-iptables-alerter-script\") pod \"iptables-alerter-tw45g\" (UID: \"8830e241-ee58-449f-9066-0fdc86670723\") " pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379170    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-system-cni-dir\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-conf-dir\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.411358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379222    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8830e241-ee58-449f-9066-0fdc86670723-host-slash\") pod \"iptables-alerter-tw45g\" (UID: \"8830e241-ee58-449f-9066-0fdc86670723\") " pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379245    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4bb9cb9e-5076-44dd-8588-290beecbc993-konnectivity-ca\") pod \"konnectivity-agent-dr2g4\" (UID: \"4bb9cb9e-5076-44dd-8588-290beecbc993\") " pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-device-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-sys-fs\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379318    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-kubelet\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379346    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-run-ovn-kubernetes\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379371    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-socket-dir-parent\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379397    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6xtx\" (UniqueName: \"kubernetes.io/projected/8830e241-ee58-449f-9066-0fdc86670723-kube-api-access-c6xtx\") pod \"iptables-alerter-tw45g\" (UID: \"8830e241-ee58-449f-9066-0fdc86670723\") " pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379419    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-var-lib-cni-multus\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379442    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbc4t\" (UniqueName: \"kubernetes.io/projected/28c0761e-a2fb-45db-9e96-a0464664e358-kube-api-access-jbc4t\") pod \"node-ca-cr2nk\" (UID: \"28c0761e-a2fb-45db-9e96-a0464664e358\") " pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-registration-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-tmp\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.415179 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-run-netns\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379643    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-cni-netd\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379670    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qncr8\" (UniqueName: \"kubernetes.io/projected/7429f1ce-5023-470a-9736-077b1ab02ff4-kube-api-access-qncr8\") pod \"node-resolver-cxg4g\" (UID: \"7429f1ce-5023-470a-9736-077b1ab02ff4\") " pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4bb9cb9e-5076-44dd-8588-290beecbc993-agent-certs\") pod \"konnectivity-agent-dr2g4\" (UID: \"4bb9cb9e-5076-44dd-8588-290beecbc993\") " pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379725    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-daemon-config\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7429f1ce-5023-470a-9736-077b1ab02ff4-tmp-dir\") pod \"node-resolver-cxg4g\" (UID: \"7429f1ce-5023-470a-9736-077b1ab02ff4\") " pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-modprobe-d\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379801    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-slash\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-var-lib-cni-bin\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379856    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-sysctl-d\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379881    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-systemd\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379905    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-run-systemd\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379930    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-ovnkube-script-lib\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379957    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whnzt\" (UniqueName: \"kubernetes.io/projected/8c1d1eff-c83c-4e12-b106-2181f688bb58-kube-api-access-whnzt\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.416003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.379981    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-cnibin\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.380014    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-sysctl-conf\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.380942    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28c0761e-a2fb-45db-9e96-a0464664e358-serviceca\") pod \"node-ca-cr2nk\" (UID: \"28c0761e-a2fb-45db-9e96-a0464664e358\") " pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.381171    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-cni-dir\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.381247    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-os-release\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.381297    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.381383    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-etc-selinux\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.381426    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-var-lib-kubelet\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.381466    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c0761e-a2fb-45db-9e96-a0464664e358-host\") pod \"node-ca-cr2nk\" (UID: \"28c0761e-a2fb-45db-9e96-a0464664e358\") " pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.381587    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-socket-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.381674    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-hostroot\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382461    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c1d1eff-c83c-4e12-b106-2181f688bb58-cni-binary-copy\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382614    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-device-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382712    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-run-netns\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382715    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-sys-fs\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.492403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382779    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-run-k8s-cni-cncf-io\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382790    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-etc-kubernetes\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382806    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-socket-dir-parent\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382870    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-system-cni-dir\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382892    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3c35b40-12db-4440-ac22-5146939e3455-registration-dir\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.382991    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-conf-dir\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.383024    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8830e241-ee58-449f-9066-0fdc86670723-host-slash\") pod \"iptables-alerter-tw45g\" (UID: \"8830e241-ee58-449f-9066-0fdc86670723\") " pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.383204    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.383280    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs podName:10ba62ec-7006-4a09-901f-2d09df92881e nodeName:}" failed. No retries permitted until 2026-07-24 14:33:37.883245533 +0000 UTC m=+3.036246245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs") pod "network-metrics-daemon-qdmlm" (UID: "10ba62ec-7006-4a09-901f-2d09df92881e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.383440    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-cnibin\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.383478    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-var-lib-cni-bin\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.383529    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-var-lib-cni-multus\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.383569    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c1d1eff-c83c-4e12-b106-2181f688bb58-host-run-multus-certs\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.383671    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4bb9cb9e-5076-44dd-8588-290beecbc993-konnectivity-ca\") pod \"konnectivity-agent-dr2g4\" (UID: \"4bb9cb9e-5076-44dd-8588-290beecbc993\") " pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.383814    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 14:33:37.493210 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.384182    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c1d1eff-c83c-4e12-b106-2181f688bb58-multus-daemon-config\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.384281    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8830e241-ee58-449f-9066-0fdc86670723-iptables-alerter-script\") pod \"iptables-alerter-tw45g\" (UID: \"8830e241-ee58-449f-9066-0fdc86670723\") " pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.392642    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4bb9cb9e-5076-44dd-8588-290beecbc993-agent-certs\") pod \"konnectivity-agent-dr2g4\" (UID: \"4bb9cb9e-5076-44dd-8588-290beecbc993\") " pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.395387    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnjr7\" (UniqueName: \"kubernetes.io/projected/10ba62ec-7006-4a09-901f-2d09df92881e-kube-api-access-tnjr7\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.406888    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.406911    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.406924    2583 projected.go:196] Error preparing data for projected volume kube-api-access-9w4cl for pod openshift-network-diagnostics/network-check-target-vghl9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.406991    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl podName:a4871ba7-53f5-4cec-a577-55024528f654 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:37.906972855 +0000 UTC m=+3.059973578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9w4cl" (UniqueName: "kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl") pod "network-check-target-vghl9" (UID: "a4871ba7-53f5-4cec-a577-55024528f654") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.407659    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whnzt\" (UniqueName: \"kubernetes.io/projected/8c1d1eff-c83c-4e12-b106-2181f688bb58-kube-api-access-whnzt\") pod \"multus-d4n8r\" (UID: \"8c1d1eff-c83c-4e12-b106-2181f688bb58\") " pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.408661    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbc4t\" (UniqueName: \"kubernetes.io/projected/28c0761e-a2fb-45db-9e96-a0464664e358-kube-api-access-jbc4t\") pod \"node-ca-cr2nk\" (UID: \"28c0761e-a2fb-45db-9e96-a0464664e358\") " pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.409985    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6xtx\" (UniqueName: \"kubernetes.io/projected/8830e241-ee58-449f-9066-0fdc86670723-kube-api-access-c6xtx\") pod \"iptables-alerter-tw45g\" (UID: \"8830e241-ee58-449f-9066-0fdc86670723\") " pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.410325    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbc7d\" (UniqueName: \"kubernetes.io/projected/a3c35b40-12db-4440-ac22-5146939e3455-kube-api-access-cbc7d\") pod \"aws-ebs-csi-driver-node-ltvkw\" (UID: \"a3c35b40-12db-4440-ac22-5146939e3455\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480732    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-kubelet\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480766    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-run-ovn-kubernetes\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.499328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-tmp\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480850    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-run-netns\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480852    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-kubelet\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-cni-netd\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480895    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qncr8\" (UniqueName: \"kubernetes.io/projected/7429f1ce-5023-470a-9736-077b1ab02ff4-kube-api-access-qncr8\") pod \"node-resolver-cxg4g\" (UID: \"7429f1ce-5023-470a-9736-077b1ab02ff4\") " pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480919    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7429f1ce-5023-470a-9736-077b1ab02ff4-tmp-dir\") pod \"node-resolver-cxg4g\" (UID: \"7429f1ce-5023-470a-9736-077b1ab02ff4\") " pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-modprobe-d\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480961    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-slash\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.480982    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-sysctl-d\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481002    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-systemd\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481013    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481031    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-run-systemd\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-ovnkube-script-lib\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481076    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-cnibin\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.502679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481098    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-sysctl-conf\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481120    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-sys\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481126    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-sysctl-d\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481135    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-modprobe-d\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481140    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-lib-modules\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481162    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-run-ovn-kubernetes\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481195    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481224    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-lib-modules\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-ovnkube-config\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfsgk\" (UniqueName: \"kubernetes.io/projected/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-kube-api-access-pfsgk\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481288    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d8c42ee6-258a-4618-93b7-d8781893e27f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481314    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d8c42ee6-258a-4618-93b7-d8781893e27f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481338    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxk89\" (UniqueName: \"kubernetes.io/projected/d8c42ee6-258a-4618-93b7-d8781893e27f-kube-api-access-qxk89\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481355    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7429f1ce-5023-470a-9736-077b1ab02ff4-tmp-dir\") pod \"node-resolver-cxg4g\" (UID: \"7429f1ce-5023-470a-9736-077b1ab02ff4\") " pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.503742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481395    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-run-openvswitch\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-run-openvswitch\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481434    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-cnibin\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481439    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-var-lib-kubelet\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481514    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-var-lib-kubelet\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481551    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-run-systemd\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481560    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-sysctl-conf\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/585bbc0a-c23e-4e78-b089-9e41147e92c9-dbus\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481593    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-systemd\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481611    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-systemd-units\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-run-ovn\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-log-socket\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481681    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-kubernetes\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-run\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481723    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-ovn-node-metrics-cert\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.504981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481752    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-os-release\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481773    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-ovnkube-script-lib\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481780    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-ovnkube-config\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7429f1ce-5023-470a-9736-077b1ab02ff4-hosts-file\") pod \"node-resolver-cxg4g\" (UID: \"7429f1ce-5023-470a-9736-077b1ab02ff4\") " pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481815    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7429f1ce-5023-470a-9736-077b1ab02ff4-hosts-file\") pod \"node-resolver-cxg4g\" (UID: \"7429f1ce-5023-470a-9736-077b1ab02ff4\") " pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481826    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-host\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481839    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-slash\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481852    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-tuned\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-sys\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481878    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d8c42ee6-258a-4618-93b7-d8781893e27f-cni-binary-copy\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481902    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-etc-openvswitch\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.481917    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481926    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/585bbc0a-c23e-4e78-b089-9e41147e92c9-kubelet-config\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-var-lib-openvswitch\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.506049 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.481966    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret podName:585bbc0a-c23e-4e78-b089-9e41147e92c9 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:37.981950325 +0000 UTC m=+3.134951036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret") pod "global-pull-secret-syncer-lntn6" (UID: "585bbc0a-c23e-4e78-b089-9e41147e92c9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481991    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-var-lib-openvswitch\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481997    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-cni-bin\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482025    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-env-overrides\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482032    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-etc-openvswitch\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482067    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-system-cni-dir\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482070    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/585bbc0a-c23e-4e78-b089-9e41147e92c9-kubelet-config\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481998    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-run-netns\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-sysconfig\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482097    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-run\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xndgm\" (UniqueName: \"kubernetes.io/projected/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-kube-api-access-xndgm\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482164    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-sysconfig\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.481995    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-run-ovn\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482276    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-cni-bin\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482314    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-system-cni-dir\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.507102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482327    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-kubernetes\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482162    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-node-log\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482354    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d8c42ee6-258a-4618-93b7-d8781893e27f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482379    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-host\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482437    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-node-log\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482442    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/585bbc0a-c23e-4e78-b089-9e41147e92c9-dbus\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482474    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d8c42ee6-258a-4618-93b7-d8781893e27f-os-release\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482504    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-systemd-units\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482515    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-cni-netd\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482537    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-env-overrides\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482583    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482584    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-log-socket\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482868    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d8c42ee6-258a-4618-93b7-d8781893e27f-cni-binary-copy\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.507975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.482918    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d8c42ee6-258a-4618-93b7-d8781893e27f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.509039 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.484396    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-etc-tuned\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.509039 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.484864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-tmp\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.509039 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.486009    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-ovn-node-metrics-cert\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.509039 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.490769    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxk89\" (UniqueName: \"kubernetes.io/projected/d8c42ee6-258a-4618-93b7-d8781893e27f-kube-api-access-qxk89\") pod \"multus-additional-cni-plugins-9rw82\" (UID: \"d8c42ee6-258a-4618-93b7-d8781893e27f\") " pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.509039 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.490812    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xndgm\" (UniqueName: \"kubernetes.io/projected/4f24a00a-6a4a-41f5-8257-a3194b11fd6a-kube-api-access-xndgm\") pod \"tuned-b6n7q\" (UID: \"4f24a00a-6a4a-41f5-8257-a3194b11fd6a\") " pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.509039 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.491151    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qncr8\" (UniqueName: \"kubernetes.io/projected/7429f1ce-5023-470a-9736-077b1ab02ff4-kube-api-access-qncr8\") pod \"node-resolver-cxg4g\" (UID: \"7429f1ce-5023-470a-9736-077b1ab02ff4\") " pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.509039 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.492053    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfsgk\" (UniqueName: \"kubernetes.io/projected/fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b-kube-api-access-pfsgk\") pod \"ovnkube-node-vqqjv\" (UID: \"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.576097 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.576076    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tw45g"
Jul 24 14:33:37.585643 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.585620    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:33:37.587680 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:37.587649    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8830e241_ee58_449f_9066_0fdc86670723.slice/crio-c2e0f7b82e97729310925ba130d48ce97cf65e040e821a52a8e27a26275d6c02 WatchSource:0}: Error finding container c2e0f7b82e97729310925ba130d48ce97cf65e040e821a52a8e27a26275d6c02: Status 404 returned error can't find the container with id c2e0f7b82e97729310925ba130d48ce97cf65e040e821a52a8e27a26275d6c02
Jul 24 14:33:37.597244 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.597223    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:37.607610 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.607587    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cr2nk"
Jul 24 14:33:37.613276 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.613258    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d4n8r"
Jul 24 14:33:37.622189 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.622170    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw"
Jul 24 14:33:37.628888 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.628866    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9rw82"
Jul 24 14:33:37.636976 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.636839    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cxg4g"
Jul 24 14:33:37.646634 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.646494    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b6n7q"
Jul 24 14:33:37.726861 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.726838    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:33:37.887032 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.886956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:37.887161 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.887094    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:37.887211 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.887186    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs podName:10ba62ec-7006-4a09-901f-2d09df92881e nodeName:}" failed. No retries permitted until 2026-07-24 14:33:38.887167221 +0000 UTC m=+4.040167928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs") pod "network-metrics-daemon-qdmlm" (UID: "10ba62ec-7006-4a09-901f-2d09df92881e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:37.987780 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.987747    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:37.987911 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:37.987826    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:37.987977 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.987904    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:33:37.987977 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.987929    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:33:37.987977 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.987942    2583 projected.go:196] Error preparing data for projected volume kube-api-access-9w4cl for pod openshift-network-diagnostics/network-check-target-vghl9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:37.987977 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.987960    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:37.988122 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.988000    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl podName:a4871ba7-53f5-4cec-a577-55024528f654 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:38.987979686 +0000 UTC m=+4.140980391 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w4cl" (UniqueName: "kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl") pod "network-check-target-vghl9" (UID: "a4871ba7-53f5-4cec-a577-55024528f654") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:37.988122 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:37.988029    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret podName:585bbc0a-c23e-4e78-b089-9e41147e92c9 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:38.988010008 +0000 UTC m=+4.141010714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret") pod "global-pull-secret-syncer-lntn6" (UID: "585bbc0a-c23e-4e78-b089-9e41147e92c9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:38.315787 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:38.315696    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 14:28:37 +0000 UTC" deadline="2028-01-17 10:39:58.924182544 +0000 UTC"
Jul 24 14:33:38.315787 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:38.315750    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13004h6m20.608437094s"
Jul 24 14:33:38.354260 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:38.354224    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tw45g" event={"ID":"8830e241-ee58-449f-9066-0fdc86670723","Type":"ContainerStarted","Data":"c2e0f7b82e97729310925ba130d48ce97cf65e040e821a52a8e27a26275d6c02"}
Jul 24 14:33:38.674347 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:38.674319    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7429f1ce_5023_470a_9736_077b1ab02ff4.slice/crio-87ab6b483d918b8b72cf770b1ca1fd4cae9a6c6c70e6f8dab05608156c8e4fbc WatchSource:0}: Error finding container 87ab6b483d918b8b72cf770b1ca1fd4cae9a6c6c70e6f8dab05608156c8e4fbc: Status 404 returned error can't find the container with id 87ab6b483d918b8b72cf770b1ca1fd4cae9a6c6c70e6f8dab05608156c8e4fbc
Jul 24 14:33:38.676650 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:38.675769    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1d1eff_c83c_4e12_b106_2181f688bb58.slice/crio-beb92197111df34321492a530f6a7f83b7825044b945c981898f4b8af037f4f2 WatchSource:0}: Error finding container beb92197111df34321492a530f6a7f83b7825044b945c981898f4b8af037f4f2: Status 404 returned error can't find the container with id beb92197111df34321492a530f6a7f83b7825044b945c981898f4b8af037f4f2
Jul 24 14:33:38.685347 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:38.685320    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c42ee6_258a_4618_93b7_d8781893e27f.slice/crio-71ae24c4f1c7ebfdef3791de12d48172f19ce78e4b6f7357ee70183db43a6aed WatchSource:0}: Error finding container 71ae24c4f1c7ebfdef3791de12d48172f19ce78e4b6f7357ee70183db43a6aed: Status 404 returned error can't find the container with id 71ae24c4f1c7ebfdef3791de12d48172f19ce78e4b6f7357ee70183db43a6aed
Jul 24 14:33:38.705461 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:33:38.705415    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f24a00a_6a4a_41f5_8257_a3194b11fd6a.slice/crio-317c84835e87ea7c12e8c9f4729b8b73c5560403d82f55d0ea5b17733ff8f5a1 WatchSource:0}: Error finding container 317c84835e87ea7c12e8c9f4729b8b73c5560403d82f55d0ea5b17733ff8f5a1: Status 404 returned error can't find the container with id 317c84835e87ea7c12e8c9f4729b8b73c5560403d82f55d0ea5b17733ff8f5a1
Jul 24 14:33:38.892970 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:38.892939    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:38.893121 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:38.893062    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:38.893121 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:38.893113    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs podName:10ba62ec-7006-4a09-901f-2d09df92881e nodeName:}" failed. No retries permitted until 2026-07-24 14:33:40.893100814 +0000 UTC m=+6.046101518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs") pod "network-metrics-daemon-qdmlm" (UID: "10ba62ec-7006-4a09-901f-2d09df92881e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:38.994057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:38.993975    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:38.994057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:38.994031    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:38.994247 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:38.994127    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:38.994247 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:38.994193    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret podName:585bbc0a-c23e-4e78-b089-9e41147e92c9 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:40.994173377 +0000 UTC m=+6.147174102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret") pod "global-pull-secret-syncer-lntn6" (UID: "585bbc0a-c23e-4e78-b089-9e41147e92c9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:38.994247 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:38.994136    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:33:38.994247 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:38.994234    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:33:38.994247 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:38.994244    2583 projected.go:196] Error preparing data for projected volume kube-api-access-9w4cl for pod openshift-network-diagnostics/network-check-target-vghl9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:38.994503 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:38.994296    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl podName:a4871ba7-53f5-4cec-a577-55024528f654 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:40.994272048 +0000 UTC m=+6.147272756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w4cl" (UniqueName: "kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl") pod "network-check-target-vghl9" (UID: "a4871ba7-53f5-4cec-a577-55024528f654") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:39.316660 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.316532    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 14:28:37 +0000 UTC" deadline="2028-01-01 00:55:56.600812157 +0000 UTC"
Jul 24 14:33:39.316660 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.316601    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12610h22m17.284215214s"
Jul 24 14:33:39.337585 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.337561    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:39.337702 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:39.337680    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:39.338088 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.338070    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:39.338198 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:39.338178    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:39.338293 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.338277    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:39.338380 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:39.338362    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:39.356405 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.356381    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cr2nk" event={"ID":"28c0761e-a2fb-45db-9e96-a0464664e358","Type":"ContainerStarted","Data":"dfb5a76ed61971188afc923366cbf8a42546958f0c1f4fe6016c7db9f4c7357c"}
Jul 24 14:33:39.357389 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.357370    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" event={"ID":"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b","Type":"ContainerStarted","Data":"39e99fe8f82d2ffbdda666198e57ed53d7997f3b887d03e706df72d8d2fea6be"}
Jul 24 14:33:39.358292 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.358275    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dr2g4" event={"ID":"4bb9cb9e-5076-44dd-8588-290beecbc993","Type":"ContainerStarted","Data":"b5b2a25af1a4868641e4d3cdee17b1b38a2c837d308818c78759d46277fec565"}
Jul 24 14:33:39.359261 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.359244    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d4n8r" event={"ID":"8c1d1eff-c83c-4e12-b106-2181f688bb58","Type":"ContainerStarted","Data":"beb92197111df34321492a530f6a7f83b7825044b945c981898f4b8af037f4f2"}
Jul 24 14:33:39.360416 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.360391    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b6n7q" event={"ID":"4f24a00a-6a4a-41f5-8257-a3194b11fd6a","Type":"ContainerStarted","Data":"317c84835e87ea7c12e8c9f4729b8b73c5560403d82f55d0ea5b17733ff8f5a1"}
Jul 24 14:33:39.361435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.361415    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rw82" event={"ID":"d8c42ee6-258a-4618-93b7-d8781893e27f","Type":"ContainerStarted","Data":"71ae24c4f1c7ebfdef3791de12d48172f19ce78e4b6f7357ee70183db43a6aed"}
Jul 24 14:33:39.362433 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.362416    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw" event={"ID":"a3c35b40-12db-4440-ac22-5146939e3455","Type":"ContainerStarted","Data":"fe9f492801c1c7abad157df97097ce67cf09e89915b1cb3feebdcfd1e229003c"}
Jul 24 14:33:39.363590 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:39.363570    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cxg4g" event={"ID":"7429f1ce-5023-470a-9736-077b1ab02ff4","Type":"ContainerStarted","Data":"87ab6b483d918b8b72cf770b1ca1fd4cae9a6c6c70e6f8dab05608156c8e4fbc"}
Jul 24 14:33:40.376127 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:40.375912    2583 generic.go:356] "Generic (PLEG): container finished" podID="fe1e8f4bf149517fec76eac6337ba030" containerID="8b18858cea8348ea15512f1ed406d7bfc76456356d8f04277d0cffe8b9e0481f" exitCode=0
Jul 24 14:33:40.376568 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:40.376066    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" event={"ID":"fe1e8f4bf149517fec76eac6337ba030","Type":"ContainerDied","Data":"8b18858cea8348ea15512f1ed406d7bfc76456356d8f04277d0cffe8b9e0481f"}
Jul 24 14:33:40.393064 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:40.391177    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cxg4g" event={"ID":"7429f1ce-5023-470a-9736-077b1ab02ff4","Type":"ContainerStarted","Data":"8e13bd6fbfa4cc51b1ec95774f27a6a9c527982f160a3d0461973a1a39ddafff"}
Jul 24 14:33:40.407251 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:40.406974    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cxg4g" podStartSLOduration=4.709760167 podStartE2EDuration="5.406956813s" totalImagesPullingTime="697.196646ms" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:40.406123165 +0000 UTC m=+5.559123892" watchObservedRunningTime="2026-07-24 14:33:40.406956813 +0000 UTC m=+5.559957545"
Jul 24 14:33:40.419939 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:40.419786    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" event={"ID":"1557375bdf94e26d9bcd144776d91190","Type":"ContainerStarted","Data":"7a2ca780cbe40d269ac89b717cb74b591c2f0e0e04c5ddad9f9e63acde594757"}
Jul 24 14:33:40.908751 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:40.908674    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:40.908900 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:40.908802    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:40.908900 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:40.908862    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs podName:10ba62ec-7006-4a09-901f-2d09df92881e nodeName:}" failed. No retries permitted until 2026-07-24 14:33:44.90884355 +0000 UTC m=+10.061844254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs") pod "network-metrics-daemon-qdmlm" (UID: "10ba62ec-7006-4a09-901f-2d09df92881e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:41.009172 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.009139    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:41.009336 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.009207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:41.009395 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:41.009369    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:33:41.009395 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:41.009392    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:33:41.009622 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:41.009404    2583 projected.go:196] Error preparing data for projected volume kube-api-access-9w4cl for pod openshift-network-diagnostics/network-check-target-vghl9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:41.009622 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:41.009479    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl podName:a4871ba7-53f5-4cec-a577-55024528f654 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:45.009443535 +0000 UTC m=+10.162444240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w4cl" (UniqueName: "kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl") pod "network-check-target-vghl9" (UID: "a4871ba7-53f5-4cec-a577-55024528f654") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:41.009622 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:41.009557    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:41.009622 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:41.009595    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret podName:585bbc0a-c23e-4e78-b089-9e41147e92c9 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:45.009583608 +0000 UTC m=+10.162584316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret") pod "global-pull-secret-syncer-lntn6" (UID: "585bbc0a-c23e-4e78-b089-9e41147e92c9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:41.339080 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.338987    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:41.339224 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:41.339107    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:41.339500 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.339480    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:41.339613 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:41.339572    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:41.339681 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.339631    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:41.339738 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:41.339720    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:41.429585 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.429181    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" event={"ID":"fe1e8f4bf149517fec76eac6337ba030","Type":"ContainerStarted","Data":"1ac23f23dfa32f817bbb47018cc5d0b3f143bcdb79c67c2efedf3d2585621c3e"}
Jul 24 14:33:41.433163 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.433093    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tw45g" event={"ID":"8830e241-ee58-449f-9066-0fdc86670723","Type":"ContainerStarted","Data":"16fb9638018a1c27b65bdd7c36d496b1f9d9704c6dac7b24048c648ef0853eba"}
Jul 24 14:33:41.443144 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.443092    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" podStartSLOduration=5.443073879 podStartE2EDuration="5.443073879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:40.43365537 +0000 UTC m=+5.586656099" watchObservedRunningTime="2026-07-24 14:33:41.443073879 +0000 UTC m=+6.596074606"
Jul 24 14:33:41.443262 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.443178    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" podStartSLOduration=5.44317086 podStartE2EDuration="5.44317086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:41.442218296 +0000 UTC m=+6.595219023" watchObservedRunningTime="2026-07-24 14:33:41.44317086 +0000 UTC m=+6.596171591"
Jul 24 14:33:41.454019 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:41.453978    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tw45g" podStartSLOduration=4.667498006 podStartE2EDuration="6.453962327s" totalImagesPullingTime="1.786464321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:41.453679868 +0000 UTC m=+6.606680595" watchObservedRunningTime="2026-07-24 14:33:41.453962327 +0000 UTC m=+6.606963060"
Jul 24 14:33:43.338433 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:43.338097    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:43.338433 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:43.338203    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:43.338433 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:43.338308    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:43.338433 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:43.338199    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:43.338433 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:43.338099    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:43.338433 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:43.338401    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:44.940011 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:44.939965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:44.940498 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:44.940156    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:44.940498 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:44.940224    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs podName:10ba62ec-7006-4a09-901f-2d09df92881e nodeName:}" failed. No retries permitted until 2026-07-24 14:33:52.94020555 +0000 UTC m=+18.093206258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs") pod "network-metrics-daemon-qdmlm" (UID: "10ba62ec-7006-4a09-901f-2d09df92881e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:45.041968 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:45.041160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:45.041968 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:45.041223    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:45.041968 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:45.041380    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:33:45.041968 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:45.041400    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:33:45.041968 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:45.041414    2583 projected.go:196] Error preparing data for projected volume kube-api-access-9w4cl for pod openshift-network-diagnostics/network-check-target-vghl9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:45.041968 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:45.041485    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl podName:a4871ba7-53f5-4cec-a577-55024528f654 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:53.04146791 +0000 UTC m=+18.194468628 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w4cl" (UniqueName: "kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl") pod "network-check-target-vghl9" (UID: "a4871ba7-53f5-4cec-a577-55024528f654") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:45.041968 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:45.041891    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:45.041968 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:45.041940    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret podName:585bbc0a-c23e-4e78-b089-9e41147e92c9 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:53.04192566 +0000 UTC m=+18.194926368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret") pod "global-pull-secret-syncer-lntn6" (UID: "585bbc0a-c23e-4e78-b089-9e41147e92c9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:45.338314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:45.338198    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:45.338314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:45.338307    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:45.338548 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:45.338447    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:45.338548 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:45.338310    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:45.338548 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:45.338530    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:45.338700 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:45.338611    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:47.338143 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:47.338115    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:47.338606 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:47.338117    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:47.338606 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:47.338250    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:47.338606 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:47.338258    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:47.338606 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:47.338340    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:47.338606 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:47.338424    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:49.337217 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:49.337185    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:49.337618 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:49.337242    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:49.337618 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:49.337252    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:49.337618 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:49.337327    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:49.337618 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:49.337416    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:49.337618 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:49.337483    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:51.336905 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:51.336725    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:51.337330 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:51.336727    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:51.337330 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:51.337008    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:51.337330 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:51.337120    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:51.337330 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:51.336738    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:51.337330 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:51.337212    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:53.004655 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:53.004615    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:53.005105 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.004797    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:53.005105 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.004870    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs podName:10ba62ec-7006-4a09-901f-2d09df92881e nodeName:}" failed. No retries permitted until 2026-07-24 14:34:09.00485099 +0000 UTC m=+34.157851694 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs") pod "network-metrics-daemon-qdmlm" (UID: "10ba62ec-7006-4a09-901f-2d09df92881e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:33:53.105640 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:53.105609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:53.105808 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:53.105657    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:53.105808 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.105742    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:53.105808 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.105769    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:33:53.105808 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.105791    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:33:53.105808 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.105797    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret podName:585bbc0a-c23e-4e78-b089-9e41147e92c9 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:09.105781088 +0000 UTC m=+34.258781799 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret") pod "global-pull-secret-syncer-lntn6" (UID: "585bbc0a-c23e-4e78-b089-9e41147e92c9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:33:53.105808 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.105803    2583 projected.go:196] Error preparing data for projected volume kube-api-access-9w4cl for pod openshift-network-diagnostics/network-check-target-vghl9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:53.106105 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.105842    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl podName:a4871ba7-53f5-4cec-a577-55024528f654 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:09.10583054 +0000 UTC m=+34.258831244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w4cl" (UniqueName: "kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl") pod "network-check-target-vghl9" (UID: "a4871ba7-53f5-4cec-a577-55024528f654") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:33:53.336830 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:53.336750    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:53.336830 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:53.336784    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:53.337017 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.336852    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:53.337017 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:53.336931    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:53.337121 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.337039    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:53.337121 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:53.337110    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:55.337152 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:55.337117    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:55.337583 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:55.337213    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:55.337583 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:55.337302    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:55.337583 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:55.337336    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:55.337583 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:55.337443    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:55.337583 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:55.337558    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:56.473661 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.473227    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cr2nk" event={"ID":"28c0761e-a2fb-45db-9e96-a0464664e358","Type":"ContainerStarted","Data":"c58f28a4dc1c966217f7761cf199e0777826946b8be6ce6b2bd0c88b85d32bee"}
Jul 24 14:33:56.476347 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.476313    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" event={"ID":"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b","Type":"ContainerStarted","Data":"564121c12a9de2020ef7df5161259a763ace18f743e3135781310592ce452048"}
Jul 24 14:33:56.476557 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.476384    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" event={"ID":"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b","Type":"ContainerStarted","Data":"3bfcbf2dd9efbe71d5f9b181aacda6308082f474b719c92e66b1346999ccc47d"}
Jul 24 14:33:56.476557 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.476433    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" event={"ID":"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b","Type":"ContainerStarted","Data":"018c1817ef5363303e2e714ed7b89e3b16dc5d4d34c64101a66b76646e58538e"}
Jul 24 14:33:56.476557 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.476499    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" event={"ID":"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b","Type":"ContainerStarted","Data":"b6bede86a32086773949d96b9fd570f740ecba4837ded2a796ee29bf7dea585b"}
Jul 24 14:33:56.476557 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.476539    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" event={"ID":"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b","Type":"ContainerStarted","Data":"71446768f2187e68c9ee209e43d97a0238d846d7e13fefd724a9f50523e1b66e"}
Jul 24 14:33:56.476769 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.476567    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" event={"ID":"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b","Type":"ContainerStarted","Data":"8e8a3ca7223e46cec24daff8864b92efbfe1d45bb99641b90798bd1622bf454f"}
Jul 24 14:33:56.478184 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.477969    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dr2g4" event={"ID":"4bb9cb9e-5076-44dd-8588-290beecbc993","Type":"ContainerStarted","Data":"5a0a5d33eaf0d27ec91cdafa98231f2f436741c7548f4b8ecdba90e63300bafd"}
Jul 24 14:33:56.479618 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.479591    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d4n8r" event={"ID":"8c1d1eff-c83c-4e12-b106-2181f688bb58","Type":"ContainerStarted","Data":"a26c1a4c76ea0e18d4e34d853c806c36109de1b53b7b51ae226ac20dd67727e2"}
Jul 24 14:33:56.481043 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.481022    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b6n7q" event={"ID":"4f24a00a-6a4a-41f5-8257-a3194b11fd6a","Type":"ContainerStarted","Data":"c1a2328ccfe135b5b4d397bd005bd3339937847e611f7b2ff2c3105028216044"}
Jul 24 14:33:56.482653 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.482633    2583 generic.go:356] "Generic (PLEG): container finished" podID="d8c42ee6-258a-4618-93b7-d8781893e27f" containerID="f8105be6bdf12191aa3c142e7056f49d8fb62496c5ae511cef2e263ebfbcfa5a" exitCode=0
Jul 24 14:33:56.482744 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.482707    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rw82" event={"ID":"d8c42ee6-258a-4618-93b7-d8781893e27f","Type":"ContainerDied","Data":"f8105be6bdf12191aa3c142e7056f49d8fb62496c5ae511cef2e263ebfbcfa5a"}
Jul 24 14:33:56.484618 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.484557    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw" event={"ID":"a3c35b40-12db-4440-ac22-5146939e3455","Type":"ContainerStarted","Data":"c5af252da304d342c3ba068dbe15e239550be1288a0fb37d2d836b34b6b5c5e5"}
Jul 24 14:33:56.487874 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.487828    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cr2nk" podStartSLOduration=4.360657402 podStartE2EDuration="21.487811981s" totalImagesPullingTime="17.127154579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:56.487135474 +0000 UTC m=+21.640136200" watchObservedRunningTime="2026-07-24 14:33:56.487811981 +0000 UTC m=+21.640812707"
Jul 24 14:33:56.501153 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.501102    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d4n8r" podStartSLOduration=4.121263045 podStartE2EDuration="21.50109188s" totalImagesPullingTime="17.379828835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:56.500963799 +0000 UTC m=+21.653964525" watchObservedRunningTime="2026-07-24 14:33:56.50109188 +0000 UTC m=+21.654092605"
Jul 24 14:33:56.547181 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.547139    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b6n7q" podStartSLOduration=4.408134122 podStartE2EDuration="21.547122586s" totalImagesPullingTime="17.138988464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:56.54674124 +0000 UTC m=+21.699741968" watchObservedRunningTime="2026-07-24 14:33:56.547122586 +0000 UTC m=+21.700123313"
Jul 24 14:33:56.575595 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:56.575542    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dr2g4" podStartSLOduration=8.794493006 podStartE2EDuration="21.575529564s" totalImagesPullingTime="12.781036558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:56.575146271 +0000 UTC m=+21.728146996" watchObservedRunningTime="2026-07-24 14:33:56.575529564 +0000 UTC m=+21.728530289"
Jul 24 14:33:57.182739 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:57.182573    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 14:33:57.328198 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:57.328111    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T14:33:57.182736601Z","UUID":"316fd43a-5306-4680-872a-d92d670e871e","Handler":null,"Name":"","Endpoint":""}
Jul 24 14:33:57.329434 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:57.329416    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 14:33:57.329539 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:57.329474    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 14:33:57.336954 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:57.336936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:57.337062 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:57.336936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:57.337062 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:57.337056    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:57.337062 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:57.337049    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:57.337210 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:57.337116    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:57.337210 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:57.337176    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:57.489359 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:57.489293    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw" event={"ID":"a3c35b40-12db-4440-ac22-5146939e3455","Type":"ContainerStarted","Data":"257c58ce0844cd37f908d86e5e9f928d85605a84bc681ebf8b40fae51e5c0556"}
Jul 24 14:33:58.494710 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:58.494686    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw" event={"ID":"a3c35b40-12db-4440-ac22-5146939e3455","Type":"ContainerStarted","Data":"2c32895d9ddbe2467c8ec1c2a8d904935fcf6fa2726b3fd745986c10d54f6837"}
Jul 24 14:33:58.524917 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:58.524879    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltvkw" podStartSLOduration=4.357901924 podStartE2EDuration="23.524867371s" totalImagesPullingTime="19.166965447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:58.524767365 +0000 UTC m=+23.677768092" watchObservedRunningTime="2026-07-24 14:33:58.524867371 +0000 UTC m=+23.677868096"
Jul 24 14:33:58.555737 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:58.555713    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:58.556434 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:58.556415    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:59.337509 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:59.337476    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:33:59.337771 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:59.337476    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:33:59.337771 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:59.337598    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:33:59.337771 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:59.337659    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:33:59.337771 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:59.337758    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:33:59.337977 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:33:59.337844    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:33:59.499871 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:59.499839    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" event={"ID":"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b","Type":"ContainerStarted","Data":"3cd82290c17d5b8f12106d0b98f57b7eef0c571cf372e33584e4ed91e11fe495"}
Jul 24 14:33:59.500245 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:59.500088    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:33:59.500555 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:33:59.500526    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dr2g4"
Jul 24 14:34:01.336905 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.336781    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:34:01.337481 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.336781    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:34:01.337481 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:01.336974    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:34:01.337481 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:01.337066    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:34:01.337481 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.336838    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:01.337481 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:01.337165    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:34:01.505632 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.505608    2583 generic.go:356] "Generic (PLEG): container finished" podID="d8c42ee6-258a-4618-93b7-d8781893e27f" containerID="6ce88cb4d6f090a5f853c705c77ed617213e73b561e14e7127624b5a183b403b" exitCode=0
Jul 24 14:34:01.505731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.505662    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rw82" event={"ID":"d8c42ee6-258a-4618-93b7-d8781893e27f","Type":"ContainerDied","Data":"6ce88cb4d6f090a5f853c705c77ed617213e73b561e14e7127624b5a183b403b"}
Jul 24 14:34:01.509006 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.508986    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" event={"ID":"fe9bcaf2-3125-4e20-8e0b-1bbf45ba060b","Type":"ContainerStarted","Data":"fb074d2b7873914ada465c12beeb1aea6f0c30a2a88236b566df1c0f35aa104a"}
Jul 24 14:34:01.509253 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.509222    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:34:01.509340 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.509295    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:34:01.509388 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.509343    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:34:01.522920 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.522869    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:34:01.523007 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.522968    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:34:01.558752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:01.558715    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv" podStartSLOduration=9.348347184 podStartE2EDuration="26.558703464s" totalImagesPullingTime="17.21035628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:01.557387538 +0000 UTC m=+26.710388263" watchObservedRunningTime="2026-07-24 14:34:01.558703464 +0000 UTC m=+26.711704192"
Jul 24 14:34:03.046223 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:03.046162    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vghl9"]
Jul 24 14:34:03.046580 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:03.046296    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:03.046580 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:03.046369    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:34:03.049074 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:03.049041    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lntn6"]
Jul 24 14:34:03.049206 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:03.049142    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:34:03.049271 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:03.049213    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:34:03.051797 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:03.051772    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qdmlm"]
Jul 24 14:34:03.051917 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:03.051884    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:34:03.052008 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:03.051987    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:34:03.514420 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:03.514391    2583 generic.go:356] "Generic (PLEG): container finished" podID="d8c42ee6-258a-4618-93b7-d8781893e27f" containerID="c8a9bc54839982ea52a7b0b53dfdaf3acbf1d22725031f394c49b1145cc695e6" exitCode=0
Jul 24 14:34:03.514593 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:03.514428    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rw82" event={"ID":"d8c42ee6-258a-4618-93b7-d8781893e27f","Type":"ContainerDied","Data":"c8a9bc54839982ea52a7b0b53dfdaf3acbf1d22725031f394c49b1145cc695e6"}
Jul 24 14:34:05.342162 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:05.342139    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:34:05.342494 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:05.342228    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:05.342494 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:05.342248    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:34:05.342494 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:05.342256    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:34:05.342494 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:05.342332    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:34:05.342494 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:05.342390    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:34:05.521097 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:05.521042    2583 generic.go:356] "Generic (PLEG): container finished" podID="d8c42ee6-258a-4618-93b7-d8781893e27f" containerID="93af16252efc5be83396b14b45adb2be498227310597fba086992405039a329e" exitCode=0
Jul 24 14:34:05.521097 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:05.521078    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rw82" event={"ID":"d8c42ee6-258a-4618-93b7-d8781893e27f","Type":"ContainerDied","Data":"93af16252efc5be83396b14b45adb2be498227310597fba086992405039a329e"}
Jul 24 14:34:07.337188 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:07.336927    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:34:07.337615 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:07.336929    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:34:07.337615 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:07.337256    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:34:07.337615 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:07.336960    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:07.337615 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:07.337323    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lntn6" podUID="585bbc0a-c23e-4e78-b089-9e41147e92c9"
Jul 24 14:34:07.337615 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:07.337401    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vghl9" podUID="a4871ba7-53f5-4cec-a577-55024528f654"
Jul 24 14:34:08.676236 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.676205    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeReady"
Jul 24 14:34:08.676696 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.676331    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 14:34:08.715271 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.715240    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"]
Jul 24 14:34:08.720485 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.720415    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56d749f888-5dfwn"]
Jul 24 14:34:08.720599 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.720557    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.723827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.723807    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 14:34:08.723827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.723824    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 14:34:08.723993 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.723844    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 14:34:08.724045 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.724036    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"]
Jul 24 14:34:08.724146 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.724128    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 24 14:34:08.724220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.724156    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.726498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.726352    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 14:34:08.726498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.726376    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pzwk4\""
Jul 24 14:34:08.726498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.726388    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 14:34:08.726869 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.726853    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 14:34:08.728004 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.727952    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"]
Jul 24 14:34:08.728086 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.728058    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.731067 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.731045    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 24 14:34:08.731725 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.731392    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"
Jul 24 14:34:08.731822 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.731786    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 24 14:34:08.732151 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.731989    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 24 14:34:08.732151 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.732010    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 24 14:34:08.732676 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.732656    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 14:34:08.733071 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.733039    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qddgc"]
Jul 24 14:34:08.733905 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.733889    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-zk4hm\""
Jul 24 14:34:08.734001 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.733931    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 24 14:34:08.736551 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.736509    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"]
Jul 24 14:34:08.736551 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.736538    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"]
Jul 24 14:34:08.736699 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.736604    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qddgc"
Jul 24 14:34:08.739827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.739147    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n9c9d\""
Jul 24 14:34:08.739827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.739439    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 14:34:08.739827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.739725    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 14:34:08.740017 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.739924    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 14:34:08.740546 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.740223    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d749f888-5dfwn"]
Jul 24 14:34:08.740967 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.740945    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z7b8d"]
Jul 24 14:34:08.744635 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.744613    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qddgc"]
Jul 24 14:34:08.744720 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.744700    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.748402 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.748247    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 14:34:08.748402 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.748323    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-26zbb\""
Jul 24 14:34:08.748578 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.748469    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 14:34:08.751355 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.751335    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"]
Jul 24 14:34:08.758256 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.758231    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z7b8d"]
Jul 24 14:34:08.830864 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.830834    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6aec5749-5abe-40e4-b43a-418e240e0669-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5789b67588-vh25j\" (UID: \"6aec5749-5abe-40e4-b43a-418e240e0669\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"
Jul 24 14:34:08.830975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.830895    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwwnb\" (UniqueName: \"kubernetes.io/projected/04725f4d-6048-4701-a194-7a45ec82c7ea-kube-api-access-xwwnb\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.830975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.830936    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-trusted-ca\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.831070 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.830972    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpmjq\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-kube-api-access-zpmjq\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.831070 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831025    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.831145 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.831145 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6fx7\" (UniqueName: \"kubernetes.io/projected/5aac729c-b0e1-43cf-9606-90eedeb078fa-kube-api-access-r6fx7\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.831145 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831128    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7qcx\" (UniqueName: \"kubernetes.io/projected/f73843df-70d8-49f8-942e-9b9af5115170-kube-api-access-k7qcx\") pod \"ingress-canary-qddgc\" (UID: \"f73843df-70d8-49f8-942e-9b9af5115170\") " pod="openshift-ingress-canary/ingress-canary-qddgc"
Jul 24 14:34:08.831235 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831149    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npkkl\" (UniqueName: \"kubernetes.io/projected/5b90ab94-b5f5-4bf2-bb21-416c73ccf50a-kube-api-access-npkkl\") pod \"klusterlet-addon-workmgr-669bf98999-q8hgp\" (UID: \"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.831235 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831163    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.831235 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831181    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aac729c-b0e1-43cf-9606-90eedeb078fa-additional-service-ca\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.831235 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831202    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f73843df-70d8-49f8-942e-9b9af5115170-cert\") pod \"ingress-canary-qddgc\" (UID: \"f73843df-70d8-49f8-942e-9b9af5115170\") " pod="openshift-ingress-canary/ingress-canary-qddgc"
Jul 24 14:34:08.831235 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831223    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/04725f4d-6048-4701-a194-7a45ec82c7ea-config-volume\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.831425 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831283    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-ca\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.831425 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831300    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-image-registry-private-configuration\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.831425 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831322    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.831425 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04725f4d-6048-4701-a194-7a45ec82c7ea-tmp-dir\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.831425 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5b90ab94-b5f5-4bf2-bb21-416c73ccf50a-klusterlet-config\") pod \"klusterlet-addon-workmgr-669bf98999-q8hgp\" (UID: \"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.831425 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831397    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-registry-certificates\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.831685 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831429    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-hub\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.831685 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831449    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f04803-5077-4490-afa2-f0564f500587-ca-trust-extracted\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.831685 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831481    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b90ab94-b5f5-4bf2-bb21-416c73ccf50a-tmp\") pod \"klusterlet-addon-workmgr-669bf98999-q8hgp\" (UID: \"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.831685 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831504    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-installation-pull-secrets\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.831685 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831523    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-bound-sa-token\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.831685 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.831537    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-526n7\" (UniqueName: \"kubernetes.io/projected/6aec5749-5abe-40e4-b43a-418e240e0669-kube-api-access-526n7\") pod \"managed-serviceaccount-addon-agent-5789b67588-vh25j\" (UID: \"6aec5749-5abe-40e4-b43a-418e240e0669\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"
Jul 24 14:34:08.882897 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.882853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:08.932846 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.932819    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.932956 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.932871    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.932956 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.932899    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6fx7\" (UniqueName: \"kubernetes.io/projected/5aac729c-b0e1-43cf-9606-90eedeb078fa-kube-api-access-r6fx7\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.932956 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.932924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7qcx\" (UniqueName: \"kubernetes.io/projected/f73843df-70d8-49f8-942e-9b9af5115170-kube-api-access-k7qcx\") pod \"ingress-canary-qddgc\" (UID: \"f73843df-70d8-49f8-942e-9b9af5115170\") " pod="openshift-ingress-canary/ingress-canary-qddgc"
Jul 24 14:34:08.932956 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.932951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npkkl\" (UniqueName: \"kubernetes.io/projected/5b90ab94-b5f5-4bf2-bb21-416c73ccf50a-kube-api-access-npkkl\") pod \"klusterlet-addon-workmgr-669bf98999-q8hgp\" (UID: \"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.933155 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:08.932970    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:34:08.933155 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.932975    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.933155 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933003    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aac729c-b0e1-43cf-9606-90eedeb078fa-additional-service-ca\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.933155 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:08.933035    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls podName:04725f4d-6048-4701-a194-7a45ec82c7ea nodeName:}" failed. No retries permitted until 2026-07-24 14:34:09.433016362 +0000 UTC m=+34.586017080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls") pod "dns-default-z7b8d" (UID: "04725f4d-6048-4701-a194-7a45ec82c7ea") : secret "dns-default-metrics-tls" not found
Jul 24 14:34:08.933155 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933058    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f73843df-70d8-49f8-942e-9b9af5115170-cert\") pod \"ingress-canary-qddgc\" (UID: \"f73843df-70d8-49f8-942e-9b9af5115170\") " pod="openshift-ingress-canary/ingress-canary-qddgc"
Jul 24 14:34:08.933155 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/04725f4d-6048-4701-a194-7a45ec82c7ea-config-volume\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.933155 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933141    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-ca\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-image-registry-private-configuration\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933198    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933242    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04725f4d-6048-4701-a194-7a45ec82c7ea-tmp-dir\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5b90ab94-b5f5-4bf2-bb21-416c73ccf50a-klusterlet-config\") pod \"klusterlet-addon-workmgr-669bf98999-q8hgp\" (UID: \"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933314    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-registry-certificates\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:08.933335    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933344    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-hub\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f04803-5077-4490-afa2-f0564f500587-ca-trust-extracted\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933401    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b90ab94-b5f5-4bf2-bb21-416c73ccf50a-tmp\") pod \"klusterlet-addon-workmgr-669bf98999-q8hgp\" (UID: \"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-installation-pull-secrets\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933449    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-bound-sa-token\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-526n7\" (UniqueName: \"kubernetes.io/projected/6aec5749-5abe-40e4-b43a-418e240e0669-kube-api-access-526n7\") pod \"managed-serviceaccount-addon-agent-5789b67588-vh25j\" (UID: \"6aec5749-5abe-40e4-b43a-418e240e0669\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933526    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6aec5749-5abe-40e4-b43a-418e240e0669-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5789b67588-vh25j\" (UID: \"6aec5749-5abe-40e4-b43a-418e240e0669\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"
Jul 24 14:34:08.933535 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:08.933350    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d749f888-5dfwn: secret "image-registry-tls" not found
Jul 24 14:34:08.934218 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933579    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwwnb\" (UniqueName: \"kubernetes.io/projected/04725f4d-6048-4701-a194-7a45ec82c7ea-kube-api-access-xwwnb\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.934218 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:08.933616    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls podName:f1f04803-5077-4490-afa2-f0564f500587 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:09.433595598 +0000 UTC m=+34.586596303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls") pod "image-registry-56d749f888-5dfwn" (UID: "f1f04803-5077-4490-afa2-f0564f500587") : secret "image-registry-tls" not found
Jul 24 14:34:08.934218 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.933736    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aac729c-b0e1-43cf-9606-90eedeb078fa-additional-service-ca\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.934218 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.934105    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f04803-5077-4490-afa2-f0564f500587-ca-trust-extracted\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.934218 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.934134    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-registry-certificates\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.935002 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.934925    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/04725f4d-6048-4701-a194-7a45ec82c7ea-config-volume\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.935117 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.935070    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b90ab94-b5f5-4bf2-bb21-416c73ccf50a-tmp\") pod \"klusterlet-addon-workmgr-669bf98999-q8hgp\" (UID: \"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.935117 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.935087    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04725f4d-6048-4701-a194-7a45ec82c7ea-tmp-dir\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.935370 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.935325    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-trusted-ca\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.935565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.935538    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpmjq\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-kube-api-access-zpmjq\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.936344 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.936319    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-trusted-ca\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.938243 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.938212    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-image-registry-private-configuration\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.938353 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.938288    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f73843df-70d8-49f8-942e-9b9af5115170-cert\") pod \"ingress-canary-qddgc\" (UID: \"f73843df-70d8-49f8-942e-9b9af5115170\") " pod="openshift-ingress-canary/ingress-canary-qddgc"
Jul 24 14:34:08.938433 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.938401    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-ca\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.939020 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.938999    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-hub\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.939095 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.939056    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.939095 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.939087    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5b90ab94-b5f5-4bf2-bb21-416c73ccf50a-klusterlet-config\") pod \"klusterlet-addon-workmgr-669bf98999-q8hgp\" (UID: \"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.939262 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.939239    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5aac729c-b0e1-43cf-9606-90eedeb078fa-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.940029 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.940010    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6aec5749-5abe-40e4-b43a-418e240e0669-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5789b67588-vh25j\" (UID: \"6aec5749-5abe-40e4-b43a-418e240e0669\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"
Jul 24 14:34:08.940448 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.940431    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-installation-pull-secrets\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.944866 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.944844    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwwnb\" (UniqueName: \"kubernetes.io/projected/04725f4d-6048-4701-a194-7a45ec82c7ea-kube-api-access-xwwnb\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:08.944967 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.944922    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npkkl\" (UniqueName: \"kubernetes.io/projected/5b90ab94-b5f5-4bf2-bb21-416c73ccf50a-kube-api-access-npkkl\") pod \"klusterlet-addon-workmgr-669bf98999-q8hgp\" (UID: \"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:08.945046 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.945023    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7qcx\" (UniqueName: \"kubernetes.io/projected/f73843df-70d8-49f8-942e-9b9af5115170-kube-api-access-k7qcx\") pod \"ingress-canary-qddgc\" (UID: \"f73843df-70d8-49f8-942e-9b9af5115170\") " pod="openshift-ingress-canary/ingress-canary-qddgc"
Jul 24 14:34:08.946103 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.946083    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpmjq\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-kube-api-access-zpmjq\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.946318 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.946298    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6fx7\" (UniqueName: \"kubernetes.io/projected/5aac729c-b0e1-43cf-9606-90eedeb078fa-kube-api-access-r6fx7\") pod \"cluster-proxy-proxy-agent-644446468f-jb6n9\" (UID: \"5aac729c-b0e1-43cf-9606-90eedeb078fa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:08.946557 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.946540    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-526n7\" (UniqueName: \"kubernetes.io/projected/6aec5749-5abe-40e4-b43a-418e240e0669-kube-api-access-526n7\") pod \"managed-serviceaccount-addon-agent-5789b67588-vh25j\" (UID: \"6aec5749-5abe-40e4-b43a-418e240e0669\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"
Jul 24 14:34:08.947065 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.947048    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-bound-sa-token\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:08.968423 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:08.968393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:09.034119 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.034059    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:09.036378 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.036353    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:34:09.036554 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.036530    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:34:09.036646 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.036596    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs podName:10ba62ec-7006-4a09-901f-2d09df92881e nodeName:}" failed. No retries permitted until 2026-07-24 14:34:41.03657964 +0000 UTC m=+66.189580355 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs") pod "network-metrics-daemon-qdmlm" (UID: "10ba62ec-7006-4a09-901f-2d09df92881e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:34:09.059003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.058981    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:34:09.066993 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.066972    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"
Jul 24 14:34:09.084906 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.084887    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qddgc"
Jul 24 14:34:09.137087 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.137057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:34:09.137181 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.137140    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:09.137181 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.137158    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:34:09.137288 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.137218    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret podName:585bbc0a-c23e-4e78-b089-9e41147e92c9 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:41.137199175 +0000 UTC m=+66.290199888 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret") pod "global-pull-secret-syncer-lntn6" (UID: "585bbc0a-c23e-4e78-b089-9e41147e92c9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:34:09.137288 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.137278    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:34:09.137402 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.137296    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:34:09.137402 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.137308    2583 projected.go:196] Error preparing data for projected volume kube-api-access-9w4cl for pod openshift-network-diagnostics/network-check-target-vghl9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:34:09.137402 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.137359    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl podName:a4871ba7-53f5-4cec-a577-55024528f654 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:41.137345342 +0000 UTC m=+66.290346059 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w4cl" (UniqueName: "kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl") pod "network-check-target-vghl9" (UID: "a4871ba7-53f5-4cec-a577-55024528f654") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:34:09.337119 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.337039    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:34:09.337265 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.337047    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:09.337265 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.337050    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:34:09.340044 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.340021    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 14:34:09.340263 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.340239    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2pw7m\""
Jul 24 14:34:09.340263 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.340256    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 14:34:09.340412 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.340393    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d7cdd\""
Jul 24 14:34:09.340541 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.340422    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 14:34:09.340541 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.340487    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 14:34:09.439581 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.439547    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:09.439725 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.439634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:09.439725 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.439657    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:34:09.439837 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.439729    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls podName:04725f4d-6048-4701-a194-7a45ec82c7ea nodeName:}" failed. No retries permitted until 2026-07-24 14:34:10.439711121 +0000 UTC m=+35.592711847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls") pod "dns-default-z7b8d" (UID: "04725f4d-6048-4701-a194-7a45ec82c7ea") : secret "dns-default-metrics-tls" not found
Jul 24 14:34:09.439837 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.439730    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:34:09.439837 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.439744    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d749f888-5dfwn: secret "image-registry-tls" not found
Jul 24 14:34:09.439837 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:09.439782    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls podName:f1f04803-5077-4490-afa2-f0564f500587 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:10.439772613 +0000 UTC m=+35.592773317 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls") pod "image-registry-56d749f888-5dfwn" (UID: "f1f04803-5077-4490-afa2-f0564f500587") : secret "image-registry-tls" not found
Jul 24 14:34:09.640090 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.639949    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j"]
Jul 24 14:34:09.642865 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.642834    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qddgc"]
Jul 24 14:34:09.644037 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:34:09.644001    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6aec5749_5abe_40e4_b43a_418e240e0669.slice/crio-4ad7513c794207f0ad6b51e467024d52dc8173fecdc5d836069f0fbd1bbbc809 WatchSource:0}: Error finding container 4ad7513c794207f0ad6b51e467024d52dc8173fecdc5d836069f0fbd1bbbc809: Status 404 returned error can't find the container with id 4ad7513c794207f0ad6b51e467024d52dc8173fecdc5d836069f0fbd1bbbc809
Jul 24 14:34:09.645600 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.645580    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"]
Jul 24 14:34:09.646202 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:34:09.646161    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73843df_70d8_49f8_942e_9b9af5115170.slice/crio-229affa42e22faaaa18099c75f444b2687e90319da8a4a2c995dc3254a286c6c WatchSource:0}: Error finding container 229affa42e22faaaa18099c75f444b2687e90319da8a4a2c995dc3254a286c6c: Status 404 returned error can't find the container with id 229affa42e22faaaa18099c75f444b2687e90319da8a4a2c995dc3254a286c6c
Jul 24 14:34:09.649303 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:34:09.649272    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5aac729c_b0e1_43cf_9606_90eedeb078fa.slice/crio-f61b72fa1b38b88e4b2760768ac5abcc2cd9cb60fdad338d63bf95788a4aeb4e WatchSource:0}: Error finding container f61b72fa1b38b88e4b2760768ac5abcc2cd9cb60fdad338d63bf95788a4aeb4e: Status 404 returned error can't find the container with id f61b72fa1b38b88e4b2760768ac5abcc2cd9cb60fdad338d63bf95788a4aeb4e
Jul 24 14:34:09.652361 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:09.652326    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"]
Jul 24 14:34:09.655246 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:34:09.655219    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b90ab94_b5f5_4bf2_bb21_416c73ccf50a.slice/crio-5bbd841f408027ff225efd596766669662db3fd0e7882f5f746b001c105c61b4 WatchSource:0}: Error finding container 5bbd841f408027ff225efd596766669662db3fd0e7882f5f746b001c105c61b4: Status 404 returned error can't find the container with id 5bbd841f408027ff225efd596766669662db3fd0e7882f5f746b001c105c61b4
Jul 24 14:34:10.447848 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:10.447818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:10.448733 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:10.447898    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:10.448733 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:10.447964    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:34:10.448733 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:10.448037    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls podName:04725f4d-6048-4701-a194-7a45ec82c7ea nodeName:}" failed. No retries permitted until 2026-07-24 14:34:12.448014397 +0000 UTC m=+37.601015102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls") pod "dns-default-z7b8d" (UID: "04725f4d-6048-4701-a194-7a45ec82c7ea") : secret "dns-default-metrics-tls" not found
Jul 24 14:34:10.448733 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:10.448051    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:34:10.448733 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:10.448066    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d749f888-5dfwn: secret "image-registry-tls" not found
Jul 24 14:34:10.448733 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:10.448117    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls podName:f1f04803-5077-4490-afa2-f0564f500587 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:12.448102249 +0000 UTC m=+37.601102958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls") pod "image-registry-56d749f888-5dfwn" (UID: "f1f04803-5077-4490-afa2-f0564f500587") : secret "image-registry-tls" not found
Jul 24 14:34:10.547294 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:10.547219    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qddgc" event={"ID":"f73843df-70d8-49f8-942e-9b9af5115170","Type":"ContainerStarted","Data":"229affa42e22faaaa18099c75f444b2687e90319da8a4a2c995dc3254a286c6c"}
Jul 24 14:34:10.549137 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:10.549069    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp" event={"ID":"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a","Type":"ContainerStarted","Data":"5bbd841f408027ff225efd596766669662db3fd0e7882f5f746b001c105c61b4"}
Jul 24 14:34:10.553402 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:10.553277    2583 generic.go:356] "Generic (PLEG): container finished" podID="d8c42ee6-258a-4618-93b7-d8781893e27f" containerID="0ba68cb9d3846e12a2f1638bec447082c5353b61d6116f0f5308f7d9f56fe44e" exitCode=0
Jul 24 14:34:10.553402 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:10.553358    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rw82" event={"ID":"d8c42ee6-258a-4618-93b7-d8781893e27f","Type":"ContainerDied","Data":"0ba68cb9d3846e12a2f1638bec447082c5353b61d6116f0f5308f7d9f56fe44e"}
Jul 24 14:34:10.555141 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:10.555118    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j" event={"ID":"6aec5749-5abe-40e4-b43a-418e240e0669","Type":"ContainerStarted","Data":"4ad7513c794207f0ad6b51e467024d52dc8173fecdc5d836069f0fbd1bbbc809"}
Jul 24 14:34:10.557944 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:10.557919    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" event={"ID":"5aac729c-b0e1-43cf-9606-90eedeb078fa","Type":"ContainerStarted","Data":"f61b72fa1b38b88e4b2760768ac5abcc2cd9cb60fdad338d63bf95788a4aeb4e"}
Jul 24 14:34:11.577950 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:11.577916    2583 generic.go:356] "Generic (PLEG): container finished" podID="d8c42ee6-258a-4618-93b7-d8781893e27f" containerID="71ce30a316034e07a92798a5fec3a046fd800e09ce88753570f4cff8f202386b" exitCode=0
Jul 24 14:34:11.578370 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:11.577979    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rw82" event={"ID":"d8c42ee6-258a-4618-93b7-d8781893e27f","Type":"ContainerDied","Data":"71ce30a316034e07a92798a5fec3a046fd800e09ce88753570f4cff8f202386b"}
Jul 24 14:34:12.464806 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:12.464709    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:12.464985 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:12.464923    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:34:12.464985 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:12.464947    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d749f888-5dfwn: secret "image-registry-tls" not found
Jul 24 14:34:12.465092 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:12.465013    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls podName:f1f04803-5077-4490-afa2-f0564f500587 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:16.464992694 +0000 UTC m=+41.617993402 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls") pod "image-registry-56d749f888-5dfwn" (UID: "f1f04803-5077-4490-afa2-f0564f500587") : secret "image-registry-tls" not found
Jul 24 14:34:12.465092 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:12.465085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:12.465307 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:12.465206    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:34:12.465307 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:12.465263    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls podName:04725f4d-6048-4701-a194-7a45ec82c7ea nodeName:}" failed. No retries permitted until 2026-07-24 14:34:16.465248163 +0000 UTC m=+41.618248867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls") pod "dns-default-z7b8d" (UID: "04725f4d-6048-4701-a194-7a45ec82c7ea") : secret "dns-default-metrics-tls" not found
Jul 24 14:34:12.589072 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:12.588314    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rw82" event={"ID":"d8c42ee6-258a-4618-93b7-d8781893e27f","Type":"ContainerStarted","Data":"79cc19ca4acdfbfd4a1b5381941ca2f6c187a792a600c8d1b758648b37993cd7"}
Jul 24 14:34:12.613632 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:12.613273    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9rw82" podStartSLOduration=9.56160952 podStartE2EDuration="37.613258184s" totalImagesPullingTime="28.051648664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:12.6126045 +0000 UTC m=+37.765605227" watchObservedRunningTime="2026-07-24 14:34:12.613258184 +0000 UTC m=+37.766258923"
Jul 24 14:34:12.733418 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:12.733224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:12.834930 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:12.833997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:13.789581 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:13.789550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:13.859208 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:13.859176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:14.433983 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:14.433951    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:14.487731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:14.487698    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:16.496368 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:16.496334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:16.496831 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:16.496413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:16.496831 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:16.496553    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:34:16.496831 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:16.496567    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d749f888-5dfwn: secret "image-registry-tls" not found
Jul 24 14:34:16.496831 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:16.496621    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls podName:f1f04803-5077-4490-afa2-f0564f500587 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:24.496602697 +0000 UTC m=+49.649603403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls") pod "image-registry-56d749f888-5dfwn" (UID: "f1f04803-5077-4490-afa2-f0564f500587") : secret "image-registry-tls" not found
Jul 24 14:34:16.497033 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:16.497006    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:34:16.497088 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:16.497077    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls podName:04725f4d-6048-4701-a194-7a45ec82c7ea nodeName:}" failed. No retries permitted until 2026-07-24 14:34:24.497060033 +0000 UTC m=+49.650060749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls") pod "dns-default-z7b8d" (UID: "04725f4d-6048-4701-a194-7a45ec82c7ea") : secret "dns-default-metrics-tls" not found
Jul 24 14:34:17.602967 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:17.602937    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qddgc" event={"ID":"f73843df-70d8-49f8-942e-9b9af5115170","Type":"ContainerStarted","Data":"38122382fb351030ab6dd7448cf629b28897f72a0051c54268cbcc72bd3e51b3"}
Jul 24 14:34:17.604663 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:17.604642    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp" event={"ID":"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a","Type":"ContainerStarted","Data":"5fe863ef07b230235efe9e9ce83c7ebad397a58bf00314606b7a8ab7fbe08f0a"}
Jul 24 14:34:17.604844 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:17.604828    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:17.606214 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:17.606177    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j" event={"ID":"6aec5749-5abe-40e4-b43a-418e240e0669","Type":"ContainerStarted","Data":"30a6d520391c985b4c87387e229ddc4e406b905f7ce48a2111502748458570bc"}
Jul 24 14:34:17.606351 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:17.606320    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:34:17.644742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:17.644688    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qddgc" podStartSLOduration=1.896921439 podStartE2EDuration="9.64466946s" totalImagesPullingTime="7.747748021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:17.64427916 +0000 UTC m=+42.797279886" watchObservedRunningTime="2026-07-24 14:34:17.64466946 +0000 UTC m=+42.797670188"
Jul 24 14:34:17.666024 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:17.665932    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp" podStartSLOduration=10.92638879 podStartE2EDuration="18.665911587s" totalImagesPullingTime="7.739522797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:17.664826654 +0000 UTC m=+42.817827381" watchObservedRunningTime="2026-07-24 14:34:17.665911587 +0000 UTC m=+42.818912314"
Jul 24 14:34:17.682488 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:17.682412    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j" podStartSLOduration=10.933343886 podStartE2EDuration="18.682395574s" totalImagesPullingTime="7.749051688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:17.680251406 +0000 UTC m=+42.833252133" watchObservedRunningTime="2026-07-24 14:34:17.682395574 +0000 UTC m=+42.835396301"
Jul 24 14:34:18.615445 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:18.615399    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" event={"ID":"5aac729c-b0e1-43cf-9606-90eedeb078fa","Type":"ContainerStarted","Data":"d1c4208fc44de3f9e608bbb7fb61e15994f343938630081a159bdca57ca8e986"}
Jul 24 14:34:18.615775 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:18.615465    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" event={"ID":"5aac729c-b0e1-43cf-9606-90eedeb078fa","Type":"ContainerStarted","Data":"ee2f8368b62e90995b0ccee6cd10005f3016c80fb202d5351d846f3e0d4f803d"}
Jul 24 14:34:18.615775 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:18.615486    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" event={"ID":"5aac729c-b0e1-43cf-9606-90eedeb078fa","Type":"ContainerStarted","Data":"a720e544cf8ea827b5c06ea160da526f30f6508dfee4fb6492e8df822fbebdd9"}
Jul 24 14:34:18.638487 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:18.638429    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" podStartSLOduration=11.878844694 podStartE2EDuration="19.638416873s" totalImagesPullingTime="7.759572179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:18.637116013 +0000 UTC m=+43.790116738" watchObservedRunningTime="2026-07-24 14:34:18.638416873 +0000 UTC m=+43.791417635"
Jul 24 14:34:20.935306 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:20.935277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:21.045669 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:21.045638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:21.195913 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:21.195853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:21.294004 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:21.293983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:21.395167 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:21.395126    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:21.890870 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:21.890846    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:21.947324 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:21.947297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:22.288446 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:22.288397    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:22.355957 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:22.355937    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:23.091288 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:23.091260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:23.163228 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:23.163204    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:23.264411 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:23.264394    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:23.491634 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:23.491610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:23.556317 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:23.556279    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:24.290180 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:24.290156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:24.350854 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:24.350826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:24.554176 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:24.554131    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:24.554300 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:24.554183    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:24.554300 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:24.554264    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:34:24.554410 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:24.554327    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls podName:04725f4d-6048-4701-a194-7a45ec82c7ea nodeName:}" failed. No retries permitted until 2026-07-24 14:34:40.554314363 +0000 UTC m=+65.707315070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls") pod "dns-default-z7b8d" (UID: "04725f4d-6048-4701-a194-7a45ec82c7ea") : secret "dns-default-metrics-tls" not found
Jul 24 14:34:24.554410 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:24.554266    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:34:24.554410 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:24.554360    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d749f888-5dfwn: secret "image-registry-tls" not found
Jul 24 14:34:24.554549 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:24.554414    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls podName:f1f04803-5077-4490-afa2-f0564f500587 nodeName:}" failed. No retries permitted until 2026-07-24 14:34:40.554400952 +0000 UTC m=+65.707401672 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls") pod "image-registry-56d749f888-5dfwn" (UID: "f1f04803-5077-4490-afa2-f0564f500587") : secret "image-registry-tls" not found
Jul 24 14:34:24.697000 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:24.696971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:24.757292 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:24.757270    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:33.535938 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:33.535913    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqqjv"
Jul 24 14:34:33.698407 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:33.698376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:33.799113 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:33.799049    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:40.558914 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:40.558879    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:34:40.559441 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:40.558969    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:34:40.559441 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:40.559031    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:34:40.559441 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:40.559097    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:34:40.559441 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:40.559113    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d749f888-5dfwn: secret "image-registry-tls" not found
Jul 24 14:34:40.559441 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:40.559103    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls podName:04725f4d-6048-4701-a194-7a45ec82c7ea nodeName:}" failed. No retries permitted until 2026-07-24 14:35:12.559086644 +0000 UTC m=+97.712087359 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls") pod "dns-default-z7b8d" (UID: "04725f4d-6048-4701-a194-7a45ec82c7ea") : secret "dns-default-metrics-tls" not found
Jul 24 14:34:40.559441 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:40.559207    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls podName:f1f04803-5077-4490-afa2-f0564f500587 nodeName:}" failed. No retries permitted until 2026-07-24 14:35:12.559188015 +0000 UTC m=+97.712188734 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls") pod "image-registry-56d749f888-5dfwn" (UID: "f1f04803-5077-4490-afa2-f0564f500587") : secret "image-registry-tls" not found
Jul 24 14:34:41.062683 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.062660    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:34:41.065408 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.065393    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 14:34:41.073049 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:41.073034    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 14:34:41.073116 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:34:41.073080    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs podName:10ba62ec-7006-4a09-901f-2d09df92881e nodeName:}" failed. No retries permitted until 2026-07-24 14:35:45.073067567 +0000 UTC m=+130.226068272 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs") pod "network-metrics-daemon-qdmlm" (UID: "10ba62ec-7006-4a09-901f-2d09df92881e") : secret "metrics-daemon-secret" not found
Jul 24 14:34:41.163706 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.163675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:34:41.163799 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.163730    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:41.166404 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.166386    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 14:34:41.166565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.166549    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 14:34:41.176853 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.176838    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 14:34:41.177544 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.177526    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/585bbc0a-c23e-4e78-b089-9e41147e92c9-original-pull-secret\") pod \"global-pull-secret-syncer-lntn6\" (UID: \"585bbc0a-c23e-4e78-b089-9e41147e92c9\") " pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:34:41.187026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.187006    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w4cl\" (UniqueName: \"kubernetes.io/projected/a4871ba7-53f5-4cec-a577-55024528f654-kube-api-access-9w4cl\") pod \"network-check-target-vghl9\" (UID: \"a4871ba7-53f5-4cec-a577-55024528f654\") " pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:41.457783 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.457758    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2pw7m\""
Jul 24 14:34:41.460831 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.460816    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lntn6"
Jul 24 14:34:41.465974 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.465960    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:41.621422 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.621396    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lntn6"]
Jul 24 14:34:41.624411 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:34:41.624387    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585bbc0a_c23e_4e78_b089_9e41147e92c9.slice/crio-687965d50d63564f43d406d8d9e3ca51ef83114529e23ca5e27cd2dd1578f209 WatchSource:0}: Error finding container 687965d50d63564f43d406d8d9e3ca51ef83114529e23ca5e27cd2dd1578f209: Status 404 returned error can't find the container with id 687965d50d63564f43d406d8d9e3ca51ef83114529e23ca5e27cd2dd1578f209
Jul 24 14:34:41.638578 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.638557    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vghl9"]
Jul 24 14:34:41.641025 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:34:41.641002    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4871ba7_53f5_4cec_a577_55024528f654.slice/crio-a422218048139983ea5698b21a95348c85f422cfeddfac65df598bfa0d4a11cc WatchSource:0}: Error finding container a422218048139983ea5698b21a95348c85f422cfeddfac65df598bfa0d4a11cc: Status 404 returned error can't find the container with id a422218048139983ea5698b21a95348c85f422cfeddfac65df598bfa0d4a11cc
Jul 24 14:34:41.670966 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.670943    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vghl9" event={"ID":"a4871ba7-53f5-4cec-a577-55024528f654","Type":"ContainerStarted","Data":"a422218048139983ea5698b21a95348c85f422cfeddfac65df598bfa0d4a11cc"}
Jul 24 14:34:41.672000 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:41.671971    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lntn6" event={"ID":"585bbc0a-c23e-4e78-b089-9e41147e92c9","Type":"ContainerStarted","Data":"687965d50d63564f43d406d8d9e3ca51ef83114529e23ca5e27cd2dd1578f209"}
Jul 24 14:34:42.333574 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:42.333544    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:42.401519 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:42.401492    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:42.503809 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:42.503741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:43.327884 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:43.327849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:43.443819 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:43.443764    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:45.269188 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:45.269138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:45.338229 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:45.338192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:45.435724 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:45.435682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:45.499623 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:45.499590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:46.332053 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:46.332022    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:46.394427 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:46.394404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:46.456610 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:46.456584    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:46.690899 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:46.690867    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vghl9" event={"ID":"a4871ba7-53f5-4cec-a577-55024528f654","Type":"ContainerStarted","Data":"6a8f6c741bde9230bab7100f9c8fe1614263ff5ffeed4842541c5f00122cf3f1"}
Jul 24 14:34:46.691021 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:46.690994    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:34:46.692470 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:46.692437    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lntn6" event={"ID":"585bbc0a-c23e-4e78-b089-9e41147e92c9","Type":"ContainerStarted","Data":"d25964be7fff244ff6bfb2726b11ccc8cd7f21398a5b0be3f8679c733e552e2a"}
Jul 24 14:34:46.707747 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:46.707707    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vghl9" podStartSLOduration=67.542638024 podStartE2EDuration="1m11.707688292s" totalImagesPullingTime="4.165050268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:46.707025692 +0000 UTC m=+71.860026417" watchObservedRunningTime="2026-07-24 14:34:46.707688292 +0000 UTC m=+71.860689018"
Jul 24 14:34:46.721742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:46.721703    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lntn6" podStartSLOduration=67.53651602 podStartE2EDuration="1m11.721690015s" totalImagesPullingTime="4.185173995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:46.7213443 +0000 UTC m=+71.874345027" watchObservedRunningTime="2026-07-24 14:34:46.721690015 +0000 UTC m=+71.874690743"
Jul 24 14:34:49.270306 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:49.270274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:55.382280 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:55.382246    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:55.482722 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:55.482695    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:57.445180 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:57.445153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:58.680832 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:58.680727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:59.731880 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:59.731848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:59.832981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:34:59.832953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:01.405682 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:01.405656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:01.460282 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:01.460254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:01.606981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:01.606955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:01.681718 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:01.681657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:12.569534 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:12.569506    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:35:12.569954 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:12.569550    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:35:12.569954 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:12.569616    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:35:12.569954 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:12.569670    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls podName:04725f4d-6048-4701-a194-7a45ec82c7ea nodeName:}" failed. No retries permitted until 2026-07-24 14:36:16.569657936 +0000 UTC m=+161.722658640 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls") pod "dns-default-z7b8d" (UID: "04725f4d-6048-4701-a194-7a45ec82c7ea") : secret "dns-default-metrics-tls" not found
Jul 24 14:35:12.569954 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:12.569622    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:35:12.569954 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:12.569708    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d749f888-5dfwn: secret "image-registry-tls" not found
Jul 24 14:35:12.569954 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:12.569767    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls podName:f1f04803-5077-4490-afa2-f0564f500587 nodeName:}" failed. No retries permitted until 2026-07-24 14:36:16.569753964 +0000 UTC m=+161.722754668 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls") pod "image-registry-56d749f888-5dfwn" (UID: "f1f04803-5077-4490-afa2-f0564f500587") : secret "image-registry-tls" not found
Jul 24 14:35:17.698188 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:17.698160    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vghl9"
Jul 24 14:35:20.452443 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:20.452407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:20.537987 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:20.537960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:21.792201 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:21.792167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:21.893307 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:21.893281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:22.371403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:22.371380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:22.451909 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:22.451890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:43.170791 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:43.170757    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:43.270870 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:43.270843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:45.079899 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:45.079869    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:35:45.080233 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:45.079969    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 14:35:45.080233 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:45.080019    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs podName:10ba62ec-7006-4a09-901f-2d09df92881e nodeName:}" failed. No retries permitted until 2026-07-24 14:37:47.080006456 +0000 UTC m=+252.233007160 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs") pod "network-metrics-daemon-qdmlm" (UID: "10ba62ec-7006-4a09-901f-2d09df92881e") : secret "metrics-daemon-secret" not found
Jul 24 14:35:45.288743 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:45.288718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:46.125282 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.125256    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kwzjl"]
Jul 24 14:35:46.128156 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.128142    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.130844 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.130823    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 14:35:46.131104 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.131089    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 14:35:46.131157 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.131142    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 14:35:46.131192 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.131143    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 14:35:46.131771 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.131754    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-slrpn\""
Jul 24 14:35:46.139892 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.139872    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kwzjl"]
Jul 24 14:35:46.141299 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.141274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:46.231927 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.231901    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:46.287198 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.287174    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-258r7\" (UniqueName: \"kubernetes.io/projected/b9c39353-4f2e-438c-b897-294a4311bf29-kube-api-access-258r7\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.287287 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.287208    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.287613 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.287590    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.287693 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.287676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9c39353-4f2e-438c-b897-294a4311bf29-crio-socket\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.287755 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.287732    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9c39353-4f2e-438c-b897-294a4311bf29-data-volume\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.388797 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.388749    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-258r7\" (UniqueName: \"kubernetes.io/projected/b9c39353-4f2e-438c-b897-294a4311bf29-kube-api-access-258r7\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.388797 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.388779    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.388911 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.388811    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.388911 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.388828    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9c39353-4f2e-438c-b897-294a4311bf29-crio-socket\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.388911 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.388848    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9c39353-4f2e-438c-b897-294a4311bf29-data-volume\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.388911 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:46.388875    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:35:46.389025 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:46.388911    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:46.389025 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:46.388941    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:35:46.888924363 +0000 UTC m=+132.041925092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : configmap "kube-rbac-proxy" not found
Jul 24 14:35:46.389025 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:46.388974    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:35:46.888958476 +0000 UTC m=+132.041959197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:46.389025 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.388914    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9c39353-4f2e-438c-b897-294a4311bf29-crio-socket\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.389158 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.389104    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9c39353-4f2e-438c-b897-294a4311bf29-data-volume\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.397360 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.397335    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-258r7\" (UniqueName: \"kubernetes.io/projected/b9c39353-4f2e-438c-b897-294a4311bf29-kube-api-access-258r7\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.892565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.892538    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.892651 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:46.892581    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:46.892651 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:46.892636    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:35:46.892718 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:46.892651    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:46.892718 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:46.892688    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:35:47.892678053 +0000 UTC m=+133.045678757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:46.892718 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:46.892701    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:35:47.892694678 +0000 UTC m=+133.045695382 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : configmap "kube-rbac-proxy" not found
Jul 24 14:35:47.896428 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:47.896406    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cxg4g_7429f1ce-5023-470a-9736-077b1ab02ff4/dns-node-resolver/0.log"
Jul 24 14:35:47.898619 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:47.898600    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:47.898720 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:47.898655    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:47.898720 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:47.898706    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:35:47.898803 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:47.898748    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:47.898803 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:47.898765    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:35:49.898747082 +0000 UTC m=+135.051747788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : configmap "kube-rbac-proxy" not found
Jul 24 14:35:47.898803 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:47.898786    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:35:49.898775747 +0000 UTC m=+135.051776459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:48.705646 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:48.705628    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cr2nk_28c0761e-a2fb-45db-9e96-a0464664e358/node-ca/0.log"
Jul 24 14:35:49.898759 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:49.898737    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qddgc_f73843df-70d8-49f8-942e-9b9af5115170/serve-healthcheck-canary/0.log"
Jul 24 14:35:49.912875 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:49.912855    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:49.912944 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:49.912895    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:49.912981 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:49.912962    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:35:49.913019 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:49.912986    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:49.913019 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:49.913014    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:35:53.912999618 +0000 UTC m=+139.066000336 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : configmap "kube-rbac-proxy" not found
Jul 24 14:35:49.913096 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:49.913028    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:35:53.913022081 +0000 UTC m=+139.066022785 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:53.941471 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:53.941426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:53.941809 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:53.941519    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:35:53.941809 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:53.941533    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:53.941809 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:53.941584    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:36:01.941571217 +0000 UTC m=+147.094571921 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:35:53.941809 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:53.941583    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:35:53.941809 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:35:53.941610    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:36:01.941603811 +0000 UTC m=+147.094604515 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : configmap "kube-rbac-proxy" not found
Jul 24 14:35:55.552751 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:55.552723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:57.153363 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:57.153328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:57.254103 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:57.254075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:57.352919 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:57.352895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:59.327138 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:59.327112    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:59.576708 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:59.576683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:59.669491 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:35:59.669469    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:01.996337 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:01.996301    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:36:01.996684 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:01.996355    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:36:01.996684 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:36:01.996449    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:36:01.996684 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:36:01.996538    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls podName:b9c39353-4f2e-438c-b897-294a4311bf29 nodeName:}" failed. No retries permitted until 2026-07-24 14:36:17.996519042 +0000 UTC m=+163.149519762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kwzjl" (UID: "b9c39353-4f2e-438c-b897-294a4311bf29") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:36:01.996887 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:01.996865    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9c39353-4f2e-438c-b897-294a4311bf29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:36:06.981724 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:06.981686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:07.075408 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:07.075378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:07.227319 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:07.227289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:07.311187 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:07.311139    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:07.700381 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:07.700361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:07.814690 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:07.814668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:09.327246 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:09.327219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:09.398205 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:09.398183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:09.678766 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:09.678746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:09.782748 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:09.782725    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:10.660044 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:10.660018    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:10.856652 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:10.856627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:11.032744 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:11.032696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:11.141350 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:11.141328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:11.239788 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:11.239766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:11.557811 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:11.557787    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:11.743076 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:36:11.743052    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-56d749f888-5dfwn" podUID="f1f04803-5077-4490-afa2-f0564f500587"
Jul 24 14:36:11.790542 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:36:11.790517    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-z7b8d" podUID="04725f4d-6048-4701-a194-7a45ec82c7ea"
Jul 24 14:36:11.903629 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:11.903607    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:36:11.903707 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:11.903610    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:36:12.349071 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:36:12.349049    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-qdmlm" podUID="10ba62ec-7006-4a09-901f-2d09df92881e"
Jul 24 14:36:12.767530 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:12.767512    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:12.980908 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:12.980862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:13.153636 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:13.153607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:13.359866 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:13.359849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:13.554334 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:13.554312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:13.948057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:13.948036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:14.553987 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:14.553967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:14.612334 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:14.612304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:14.749968 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:14.749939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:14.810862 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:14.810816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:15.155434 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:15.155396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:15.359736 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:15.359706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:15.753133 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:15.753107    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:16.353014 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.352994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:16.590967 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.590941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:36:16.591071 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.590987    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:36:16.593057 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.593039    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04725f4d-6048-4701-a194-7a45ec82c7ea-metrics-tls\") pod \"dns-default-z7b8d\" (UID: \"04725f4d-6048-4701-a194-7a45ec82c7ea\") " pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:36:16.593199 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.593180    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"image-registry-56d749f888-5dfwn\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") " pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:36:16.707928 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.707890    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pzwk4\""
Jul 24 14:36:16.707928 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.707891    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-26zbb\""
Jul 24 14:36:16.715004 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.714984    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:36:16.715090 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.715074    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:36:16.758940 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.758912    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:16.845970 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.845944    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d749f888-5dfwn"]
Jul 24 14:36:16.848749 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:36:16.848724    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f04803_5077_4490_afa2_f0564f500587.slice/crio-c43c83f3e2e61f90f6e84123887de48467b0b1795d47edfeaf96a629f31cf818 WatchSource:0}: Error finding container c43c83f3e2e61f90f6e84123887de48467b0b1795d47edfeaf96a629f31cf818: Status 404 returned error can't find the container with id c43c83f3e2e61f90f6e84123887de48467b0b1795d47edfeaf96a629f31cf818
Jul 24 14:36:16.858860 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.858831    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z7b8d"]
Jul 24 14:36:16.863392 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:36:16.863261    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04725f4d_6048_4701_a194_7a45ec82c7ea.slice/crio-11d095bb016e43550ea8d17c05c436084b486e8c8a49430dd51b585bd1451b58 WatchSource:0}: Error finding container 11d095bb016e43550ea8d17c05c436084b486e8c8a49430dd51b585bd1451b58: Status 404 returned error can't find the container with id 11d095bb016e43550ea8d17c05c436084b486e8c8a49430dd51b585bd1451b58
Jul 24 14:36:16.919803 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.919764    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d749f888-5dfwn" event={"ID":"f1f04803-5077-4490-afa2-f0564f500587","Type":"ContainerStarted","Data":"898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c"}
Jul 24 14:36:16.919926 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.919907    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:36:16.919990 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.919959    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d749f888-5dfwn" event={"ID":"f1f04803-5077-4490-afa2-f0564f500587","Type":"ContainerStarted","Data":"c43c83f3e2e61f90f6e84123887de48467b0b1795d47edfeaf96a629f31cf818"}
Jul 24 14:36:16.920817 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.920796    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z7b8d" event={"ID":"04725f4d-6048-4701-a194-7a45ec82c7ea","Type":"ContainerStarted","Data":"11d095bb016e43550ea8d17c05c436084b486e8c8a49430dd51b585bd1451b58"}
Jul 24 14:36:16.940154 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:16.940119    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56d749f888-5dfwn" podStartSLOduration=161.940102648 podStartE2EDuration="2m41.940102648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:36:16.940102148 +0000 UTC m=+162.093102876" watchObservedRunningTime="2026-07-24 14:36:16.940102648 +0000 UTC m=+162.093103372"
Jul 24 14:36:17.161921 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.161867    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:17.263235 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.263189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:17.364419 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.364384    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:17.605979 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.605941    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp" podUID="5b90ab94-b5f5-4bf2-bb21-416c73ccf50a" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.6:8000/readyz\": dial tcp 10.132.0.6:8000: connect: connection refused"
Jul 24 14:36:17.925906 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.925873    2583 generic.go:356] "Generic (PLEG): container finished" podID="6aec5749-5abe-40e4-b43a-418e240e0669" containerID="30a6d520391c985b4c87387e229ddc4e406b905f7ce48a2111502748458570bc" exitCode=255
Jul 24 14:36:17.926084 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.925948    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j" event={"ID":"6aec5749-5abe-40e4-b43a-418e240e0669","Type":"ContainerDied","Data":"30a6d520391c985b4c87387e229ddc4e406b905f7ce48a2111502748458570bc"}
Jul 24 14:36:17.926298 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.926282    2583 scope.go:117] "RemoveContainer" containerID="30a6d520391c985b4c87387e229ddc4e406b905f7ce48a2111502748458570bc"
Jul 24 14:36:17.927779 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.927756    2583 generic.go:356] "Generic (PLEG): container finished" podID="5b90ab94-b5f5-4bf2-bb21-416c73ccf50a" containerID="5fe863ef07b230235efe9e9ce83c7ebad397a58bf00314606b7a8ab7fbe08f0a" exitCode=1
Jul 24 14:36:17.927852 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.927830    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp" event={"ID":"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a","Type":"ContainerDied","Data":"5fe863ef07b230235efe9e9ce83c7ebad397a58bf00314606b7a8ab7fbe08f0a"}
Jul 24 14:36:17.928188 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.928174    2583 scope.go:117] "RemoveContainer" containerID="5fe863ef07b230235efe9e9ce83c7ebad397a58bf00314606b7a8ab7fbe08f0a"
Jul 24 14:36:17.955269 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:17.955244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:18.002527 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.002098    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:36:18.005219 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.005191    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9c39353-4f2e-438c-b897-294a4311bf29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kwzjl\" (UID: \"b9c39353-4f2e-438c-b897-294a4311bf29\") " pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:36:18.237482 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.237372    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kwzjl"
Jul 24 14:36:18.350130 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.350087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:18.564835 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.564809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:18.574540 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.574410    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kwzjl"]
Jul 24 14:36:18.576618 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:36:18.576598    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c39353_4f2e_438c_b897_294a4311bf29.slice/crio-f193d50207223549f196272a7f588f0c3de9e85213e0f3ef01a8eb580a7a40f7 WatchSource:0}: Error finding container f193d50207223549f196272a7f588f0c3de9e85213e0f3ef01a8eb580a7a40f7: Status 404 returned error can't find the container with id f193d50207223549f196272a7f588f0c3de9e85213e0f3ef01a8eb580a7a40f7
Jul 24 14:36:18.932107 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.932078    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp" event={"ID":"5b90ab94-b5f5-4bf2-bb21-416c73ccf50a","Type":"ContainerStarted","Data":"39cb33210f84a05ef81a6a8651e5887a81c1f4afc0bc9a52c7c7aa68f73190c1"}
Jul 24 14:36:18.932392 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.932376    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:36:18.932907 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.932888    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-669bf98999-q8hgp"
Jul 24 14:36:18.933266 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.933246    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kwzjl" event={"ID":"b9c39353-4f2e-438c-b897-294a4311bf29","Type":"ContainerStarted","Data":"f193d50207223549f196272a7f588f0c3de9e85213e0f3ef01a8eb580a7a40f7"}
Jul 24 14:36:18.934840 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.934821    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5789b67588-vh25j" event={"ID":"6aec5749-5abe-40e4-b43a-418e240e0669","Type":"ContainerStarted","Data":"c6f7776a92e95a0c7abc4b62702ce3a54e5cd50439f52e716a98f01cab34b2e4"}
Jul 24 14:36:18.936378 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.936359    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z7b8d" event={"ID":"04725f4d-6048-4701-a194-7a45ec82c7ea","Type":"ContainerStarted","Data":"866549eaf9b0bfe47b264e1193bc1dbe98ab37964d4c4f0e1a572a395335ca46"}
Jul 24 14:36:18.936480 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.936400    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z7b8d" event={"ID":"04725f4d-6048-4701-a194-7a45ec82c7ea","Type":"ContainerStarted","Data":"c70c29ee60003ffb45a6724a3b7514a02c6c82574fed49998c08e6b64ef83621"}
Jul 24 14:36:18.936571 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:18.936559    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:36:19.002479 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:19.002419    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z7b8d" podStartSLOduration=129.367855873 podStartE2EDuration="2m11.002406011s" totalImagesPullingTime="1.634550138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:36:19.00118655 +0000 UTC m=+164.154187297" watchObservedRunningTime="2026-07-24 14:36:19.002406011 +0000 UTC m=+164.155406737"
Jul 24 14:36:19.149607 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:19.149412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:19.618225 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:19.618177    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:19.972270 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:19.972174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:20.365667 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:20.365632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:20.758532 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:20.758435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:21.153812 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:21.153778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:21.347662 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:21.347626    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:21.552961 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:21.552872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:21.816892 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:21.816803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:23.958116 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:23.958076    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kwzjl" event={"ID":"b9c39353-4f2e-438c-b897-294a4311bf29","Type":"ContainerStarted","Data":"ae65afb32a9234d836ea8323ad05d481c0c8da5176d8eafe4945df8f0dfb4975"}
Jul 24 14:36:24.337334 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:24.337251    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:36:24.660629 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:24.660582    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:24.723336 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:24.723299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:26.973311 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:26.973274    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kwzjl" event={"ID":"b9c39353-4f2e-438c-b897-294a4311bf29","Type":"ContainerStarted","Data":"1e00daaee5380ee3222c48a0d6067694aee50cbf1c3e01fcb9833cfb9a9f2df3"}
Jul 24 14:36:28.942051 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:28.941967    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z7b8d"
Jul 24 14:36:29.984279 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:29.984251    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kwzjl" event={"ID":"b9c39353-4f2e-438c-b897-294a4311bf29","Type":"ContainerStarted","Data":"b32e6f15c273f94380663c0d3f127caac6daafb0aa0d4f75ec36391933d50daf"}
Jul 24 14:36:30.003581 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:30.003539    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kwzjl" podStartSLOduration=32.761981973 podStartE2EDuration="44.003524464s" totalImagesPullingTime="11.241542491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:35:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:36:30.002405486 +0000 UTC m=+175.155406213" watchObservedRunningTime="2026-07-24 14:36:30.003524464 +0000 UTC m=+175.156525189"
Jul 24 14:36:32.021549 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:32.021514    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:32.156637 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:32.156609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:32.265903 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:32.265877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:34.708758 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:34.708724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:34.791413 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:34.791385    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:34.887239 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:34.887212    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:35.082990 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:35.082933    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:35.158807 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:35.158785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:36.718801 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:36.718773    2583 patch_prober.go:28] interesting pod/image-registry-56d749f888-5dfwn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 14:36:36.719118 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:36.718813    2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56d749f888-5dfwn" podUID="f1f04803-5077-4490-afa2-f0564f500587" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 14:36:37.931765 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:37.931743    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:36:38.146040 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:38.145984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:38.246197 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:38.246137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:39.664362 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:39.664333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:39.736703 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:39.736671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:41.006573 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:41.006538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:41.110408 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:41.110385    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:47.128591 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:47.128551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:47.215004 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:47.214980    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:49.060777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:49.060745    2583 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" podUID="5aac729c-b0e1-43cf-9606-90eedeb078fa" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:36:49.263141 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:49.263106    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:49.389622 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:49.389579    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:49.504914 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:49.504889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:49.755993 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:49.755929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:51.571900 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:51.571863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:51.652952 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:51.652930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:52.123012 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:52.122987    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:52.221090 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:52.221069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:52.389112 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:52.389064    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:52.552087 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:52.552067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:52.634050 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:52.634032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:53.536645 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:53.536609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:53.631086 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:53.631063    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56d749f888-5dfwn"]
Jul 24 14:36:53.644253 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:53.644232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:53.744437 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:53.744416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:53.840287 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:53.840240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:53.940367 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:53.940346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:54.142687 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:54.142665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:54.926494 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:54.926440    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:55.739786 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:55.739757    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:55.972429 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:55.972400    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:56.073679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:56.073623    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:57.811829 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:57.811799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:58.134642 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.134614    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:58.239531 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.239512    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7r9ws"]
Jul 24 14:36:58.242608 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.242592    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.245118 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.245091    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 14:36:58.245250 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.245136    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 14:36:58.245250 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.245137    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 14:36:58.245864 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.245847    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 14:36:58.245994 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.245878    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2fgzq\""
Jul 24 14:36:58.245994 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.245879    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 14:36:58.245994 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.245850    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 14:36:58.262192 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.262151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:58.358096 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.358074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:58.369016 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.368997    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f722f0c-651a-4752-91eb-745fb807f5d3-root\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.369115 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.369022    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-tls\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.369115 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.369042    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.369115 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.369072    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-wtmp\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.369211 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.369136    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f722f0c-651a-4752-91eb-745fb807f5d3-sys\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.369211 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.369157    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-textfile\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.369211 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.369181    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckl2p\" (UniqueName: \"kubernetes.io/projected/4f722f0c-651a-4752-91eb-745fb807f5d3-kube-api-access-ckl2p\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.369211 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.369198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f722f0c-651a-4752-91eb-745fb807f5d3-metrics-client-ca\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.369324 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.369218    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470022 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.469971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-wtmp\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470118 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f722f0c-651a-4752-91eb-745fb807f5d3-sys\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470118 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470048    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-textfile\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470118 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470080    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckl2p\" (UniqueName: \"kubernetes.io/projected/4f722f0c-651a-4752-91eb-745fb807f5d3-kube-api-access-ckl2p\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470118 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470107    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f722f0c-651a-4752-91eb-745fb807f5d3-metrics-client-ca\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470132    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470148    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f722f0c-651a-4752-91eb-745fb807f5d3-sys\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f722f0c-651a-4752-91eb-745fb807f5d3-root\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470146    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-wtmp\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-tls\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470222    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470247    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f722f0c-651a-4752-91eb-745fb807f5d3-root\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470662 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:36:58.470369    2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 14:36:58.470662 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:36:58.470469    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-tls podName:4f722f0c-651a-4752-91eb-745fb807f5d3 nodeName:}" failed. No retries permitted until 2026-07-24 14:36:58.970421856 +0000 UTC m=+204.123422564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-tls") pod "node-exporter-7r9ws" (UID: "4f722f0c-651a-4752-91eb-745fb807f5d3") : secret "node-exporter-tls" not found
Jul 24 14:36:58.470662 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470487    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-textfile\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470813 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470793    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f722f0c-651a-4752-91eb-745fb807f5d3-metrics-client-ca\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.470867 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.470852    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.472306 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.472287    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.484260 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.484239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:58.487393 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.487369    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckl2p\" (UniqueName: \"kubernetes.io/projected/4f722f0c-651a-4752-91eb-745fb807f5d3-kube-api-access-ckl2p\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.600990 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.600970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:58.705998 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.705971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:58.820878 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.820817    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:58.974102 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.974084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-tls\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:58.976022 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:58.975999    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f722f0c-651a-4752-91eb-745fb807f5d3-node-exporter-tls\") pod \"node-exporter-7r9ws\" (UID: \"4f722f0c-651a-4752-91eb-745fb807f5d3\") " pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:59.059938 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:59.059910    2583 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" podUID="5aac729c-b0e1-43cf-9606-90eedeb078fa" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:36:59.152333 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:59.152307    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7r9ws"
Jul 24 14:36:59.161854 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:36:59.161834    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f722f0c_651a_4752_91eb_745fb807f5d3.slice/crio-43a609f742743b42394b01c968b42e8acf5ed88e68c15390aed0584b75559496 WatchSource:0}: Error finding container 43a609f742743b42394b01c968b42e8acf5ed88e68c15390aed0584b75559496: Status 404 returned error can't find the container with id 43a609f742743b42394b01c968b42e8acf5ed88e68c15390aed0584b75559496
Jul 24 14:36:59.207329 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:59.207301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:59.305412 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:59.305390    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:59.414651 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:36:59.414600    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:00.067107 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:00.067075    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7r9ws" event={"ID":"4f722f0c-651a-4752-91eb-745fb807f5d3","Type":"ContainerStarted","Data":"43a609f742743b42394b01c968b42e8acf5ed88e68c15390aed0584b75559496"}
Jul 24 14:37:00.682821 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:00.682795    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:00.756281 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:00.756253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:01.072135 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:01.072086    2583 generic.go:356] "Generic (PLEG): container finished" podID="4f722f0c-651a-4752-91eb-745fb807f5d3" containerID="2c3f20a62251ce839defd5dc2efb6d0c1069c5688c0e2ce296fa735bef46ce34" exitCode=0
Jul 24 14:37:01.072413 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:01.072139    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7r9ws" event={"ID":"4f722f0c-651a-4752-91eb-745fb807f5d3","Type":"ContainerDied","Data":"2c3f20a62251ce839defd5dc2efb6d0c1069c5688c0e2ce296fa735bef46ce34"}
Jul 24 14:37:01.153693 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:01.153660    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:01.289021 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:01.288990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:01.386955 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:01.386935    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:01.760055 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:01.759990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:02.081514 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:02.081440    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7r9ws" event={"ID":"4f722f0c-651a-4752-91eb-745fb807f5d3","Type":"ContainerStarted","Data":"1b5735da330baffb6f0e0198904435098ff0235ded15bffe6b7be64a8d97aeb8"}
Jul 24 14:37:02.081819 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:02.081525    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7r9ws" event={"ID":"4f722f0c-651a-4752-91eb-745fb807f5d3","Type":"ContainerStarted","Data":"352181b3bec106195308780e44b2bebcf4fb0908fb900cb0a8b2b190d6ddcf29"}
Jul 24 14:37:02.123106 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:02.123063    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7r9ws" podStartSLOduration=3.110734255 podStartE2EDuration="4.123049935s" totalImagesPullingTime="951.821921ms" totalInitContainerRuntime="60.493759ms" isStatefulPod=true podCreationTimestamp="2026-07-24 14:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:37:02.122610176 +0000 UTC m=+207.275610913" watchObservedRunningTime="2026-07-24 14:37:02.123049935 +0000 UTC m=+207.276050659"
Jul 24 14:37:02.860897 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:02.860866    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:02.947529 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:02.947496    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:03.108003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:03.107977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:04.358270 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:04.358242    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:05.170892 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:05.170863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:05.235322 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:05.235296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:09.060529 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:09.060493    2583 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" podUID="5aac729c-b0e1-43cf-9606-90eedeb078fa" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:37:09.060858 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:09.060553    2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9"
Jul 24 14:37:09.060955 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:09.060929    2583 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"ee2f8368b62e90995b0ccee6cd10005f3016c80fb202d5351d846f3e0d4f803d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 24 14:37:09.060990 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:09.060973    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" podUID="5aac729c-b0e1-43cf-9606-90eedeb078fa" containerName="service-proxy" containerID="cri-o://ee2f8368b62e90995b0ccee6cd10005f3016c80fb202d5351d846f3e0d4f803d" gracePeriod=30
Jul 24 14:37:09.742773 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:09.742746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:09.844441 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:09.844412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:10.106654 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:10.106633    2583 generic.go:356] "Generic (PLEG): container finished" podID="5aac729c-b0e1-43cf-9606-90eedeb078fa" containerID="ee2f8368b62e90995b0ccee6cd10005f3016c80fb202d5351d846f3e0d4f803d" exitCode=2
Jul 24 14:37:10.106923 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:10.106669    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" event={"ID":"5aac729c-b0e1-43cf-9606-90eedeb078fa","Type":"ContainerDied","Data":"ee2f8368b62e90995b0ccee6cd10005f3016c80fb202d5351d846f3e0d4f803d"}
Jul 24 14:37:10.106923 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:10.106698    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-644446468f-jb6n9" event={"ID":"5aac729c-b0e1-43cf-9606-90eedeb078fa","Type":"ContainerStarted","Data":"1ae98d631ae0be2b613445d50952f629aac39fc9916b6447bbbb42e4ae32e854"}
Jul 24 14:37:17.202985 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:17.202950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:17.330011 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:17.329986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:18.250472 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:18.250429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:18.353398 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:18.353367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:18.648640 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:18.648614    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56d749f888-5dfwn" podUID="f1f04803-5077-4490-afa2-f0564f500587" containerName="registry" containerID="cri-o://898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c" gracePeriod=30
Jul 24 14:37:18.883269 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:18.883246    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:37:19.007894 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.007841    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f04803-5077-4490-afa2-f0564f500587-ca-trust-extracted\") pod \"f1f04803-5077-4490-afa2-f0564f500587\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") "
Jul 24 14:37:19.007894 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.007882    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-image-registry-private-configuration\") pod \"f1f04803-5077-4490-afa2-f0564f500587\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") "
Jul 24 14:37:19.008016 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.007910    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-registry-certificates\") pod \"f1f04803-5077-4490-afa2-f0564f500587\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") "
Jul 24 14:37:19.008016 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.007938    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpmjq\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-kube-api-access-zpmjq\") pod \"f1f04803-5077-4490-afa2-f0564f500587\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") "
Jul 24 14:37:19.008016 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.007975    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-trusted-ca\") pod \"f1f04803-5077-4490-afa2-f0564f500587\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") "
Jul 24 14:37:19.008016 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.008013    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-bound-sa-token\") pod \"f1f04803-5077-4490-afa2-f0564f500587\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") "
Jul 24 14:37:19.008198 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.008042    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-installation-pull-secrets\") pod \"f1f04803-5077-4490-afa2-f0564f500587\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") "
Jul 24 14:37:19.008198 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.008079    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") pod \"f1f04803-5077-4490-afa2-f0564f500587\" (UID: \"f1f04803-5077-4490-afa2-f0564f500587\") "
Jul 24 14:37:19.008299 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.008263    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f1f04803-5077-4490-afa2-f0564f500587" (UID: "f1f04803-5077-4490-afa2-f0564f500587"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:37:19.008825 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.008788    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f1f04803-5077-4490-afa2-f0564f500587" (UID: "f1f04803-5077-4490-afa2-f0564f500587"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:37:19.010665 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.010635    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f1f04803-5077-4490-afa2-f0564f500587" (UID: "f1f04803-5077-4490-afa2-f0564f500587"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:37:19.010832 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.010812    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f1f04803-5077-4490-afa2-f0564f500587" (UID: "f1f04803-5077-4490-afa2-f0564f500587"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:37:19.010832 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.010820    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f1f04803-5077-4490-afa2-f0564f500587" (UID: "f1f04803-5077-4490-afa2-f0564f500587"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:37:19.010965 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.010936    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f1f04803-5077-4490-afa2-f0564f500587" (UID: "f1f04803-5077-4490-afa2-f0564f500587"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:37:19.011009 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.010993    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-kube-api-access-zpmjq" (OuterVolumeSpecName: "kube-api-access-zpmjq") pod "f1f04803-5077-4490-afa2-f0564f500587" (UID: "f1f04803-5077-4490-afa2-f0564f500587"). InnerVolumeSpecName "kube-api-access-zpmjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:37:19.014586 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.014560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:19.017880 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.017858    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f04803-5077-4490-afa2-f0564f500587-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f1f04803-5077-4490-afa2-f0564f500587" (UID: "f1f04803-5077-4490-afa2-f0564f500587"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:37:19.108781 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.108761    2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-image-registry-private-configuration\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:37:19.108781 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.108779    2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-registry-certificates\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:37:19.108896 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.108791    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpmjq\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-kube-api-access-zpmjq\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:37:19.108896 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.108799    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1f04803-5077-4490-afa2-f0564f500587-trusted-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:37:19.108896 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.108808    2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-bound-sa-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:37:19.108896 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.108815    2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1f04803-5077-4490-afa2-f0564f500587-installation-pull-secrets\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:37:19.108896 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.108822    2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1f04803-5077-4490-afa2-f0564f500587-registry-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:37:19.108896 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.108830    2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1f04803-5077-4490-afa2-f0564f500587-ca-trust-extracted\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:37:19.122509 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.122479    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:19.133497 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.133472    2583 generic.go:356] "Generic (PLEG): container finished" podID="f1f04803-5077-4490-afa2-f0564f500587" containerID="898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c" exitCode=0
Jul 24 14:37:19.133578 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.133502    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d749f888-5dfwn" event={"ID":"f1f04803-5077-4490-afa2-f0564f500587","Type":"ContainerDied","Data":"898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c"}
Jul 24 14:37:19.133578 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.133517    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d749f888-5dfwn"
Jul 24 14:37:19.133578 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.133552    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d749f888-5dfwn" event={"ID":"f1f04803-5077-4490-afa2-f0564f500587","Type":"ContainerDied","Data":"c43c83f3e2e61f90f6e84123887de48467b0b1795d47edfeaf96a629f31cf818"}
Jul 24 14:37:19.133840 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.133590    2583 scope.go:117] "RemoveContainer" containerID="898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c"
Jul 24 14:37:19.141180 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.141164    2583 scope.go:117] "RemoveContainer" containerID="898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c"
Jul 24 14:37:19.141417 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:37:19.141397    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c\": container with ID starting with 898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c not found: ID does not exist" containerID="898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c"
Jul 24 14:37:19.141485 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.141425    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c"} err="failed to get container status \"898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c\": rpc error: code = NotFound desc = could not find container \"898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c\": container with ID starting with 898ee0581aae7cefd94060a0a5f224d8694e54e720ecb783e9f5efe663cb8a9c not found: ID does not exist"
Jul 24 14:37:19.161580 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.161554    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56d749f888-5dfwn"]
Jul 24 14:37:19.168898 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.168876    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56d749f888-5dfwn"]
Jul 24 14:37:19.340095 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:19.340047    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1f04803-5077-4490-afa2-f0564f500587" path="/var/lib/kubelet/pods/f1f04803-5077-4490-afa2-f0564f500587/volumes"
Jul 24 14:37:24.672072 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:24.672038    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:24.772829 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:24.772793    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:25.876975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:25.876950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:27.964832 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:27.964801    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:28.080272 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:28.080243    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:37.980880 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:37.980845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:38.041996 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:38.041971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:39.655617 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:39.655586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:39.721640 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:39.721613    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:47.082691 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:47.082656    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:37:47.084710 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:47.084691    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ba62ec-7006-4a09-901f-2d09df92881e-metrics-certs\") pod \"network-metrics-daemon-qdmlm\" (UID: \"10ba62ec-7006-4a09-901f-2d09df92881e\") " pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:37:47.140145 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:47.140115    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d7cdd\""
Jul 24 14:37:47.148234 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:47.148211    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qdmlm"
Jul 24 14:37:47.291569 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:47.291545    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qdmlm"]
Jul 24 14:37:47.294444 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:37:47.294421    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ba62ec_7006_4a09_901f_2d09df92881e.slice/crio-b49026307d2ab3c3e914ed98ff64d1e406a10d3bf5680647e89c022ebc546d18 WatchSource:0}: Error finding container b49026307d2ab3c3e914ed98ff64d1e406a10d3bf5680647e89c022ebc546d18: Status 404 returned error can't find the container with id b49026307d2ab3c3e914ed98ff64d1e406a10d3bf5680647e89c022ebc546d18
Jul 24 14:37:47.677983 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:47.677955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:47.774318 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:47.774284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:47.856469 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:47.856431    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:48.217679 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:48.217648    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qdmlm" event={"ID":"10ba62ec-7006-4a09-901f-2d09df92881e","Type":"ContainerStarted","Data":"b49026307d2ab3c3e914ed98ff64d1e406a10d3bf5680647e89c022ebc546d18"}
Jul 24 14:37:49.222667 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:49.222636    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qdmlm" event={"ID":"10ba62ec-7006-4a09-901f-2d09df92881e","Type":"ContainerStarted","Data":"8085e6ca76e3bbaa7aac20236c358891f50a437cf236d736f22eba3cb5c36510"}
Jul 24 14:37:49.223003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:49.222687    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qdmlm" event={"ID":"10ba62ec-7006-4a09-901f-2d09df92881e","Type":"ContainerStarted","Data":"b4476a5db96ecb05e1cb5df9f7bd7b94d0ec735b12ff5b1554baec778654e1db"}
Jul 24 14:37:49.241168 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:49.241118    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qdmlm" podStartSLOduration=253.095234404 podStartE2EDuration="4m14.241101141s" totalImagesPullingTime="1.145866737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:37:49.240975387 +0000 UTC m=+254.393976113" watchObservedRunningTime="2026-07-24 14:37:49.241101141 +0000 UTC m=+254.394101865"
Jul 24 14:37:49.354854 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:49.354824    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:49.455237 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:37:49.455209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:14.261937 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:14.261910    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:14.345061 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:14.345040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:26.217501 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:26.217436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:26.282384 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:26.282348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:29.253711 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:29.253678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:35.272305 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:35.272279    2583 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 14:38:39.721219 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:39.721162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:39.905422 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:39.905396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:40.014782 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:40.014711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:52.323009 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:52.322965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:52.399444 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:52.399417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:52.469041 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:52.469020    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:53.069083 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:53.069059    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:53.127294 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:38:53.127266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:03.302997 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:03.302965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:03.404819 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:03.404790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:03.474416 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:03.474391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:11.886909 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:11.886876    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:11.961387 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:11.961361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:28.533930 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:28.533848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:28.629581 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:28.629546    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:50.454438 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:50.454404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:51.284828 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:51.284800    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:54.276168 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:54.276137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:54.335496 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:54.335466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:57.021569 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:57.021526    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:57.119516 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:57.119489    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:57.727730 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:57.727700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:57.833031 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:57.833011    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:59.192205 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.192180    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"]
Jul 24 14:39:59.192549 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.192375    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1f04803-5077-4490-afa2-f0564f500587" containerName="registry"
Jul 24 14:39:59.192549 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.192385    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1f04803-5077-4490-afa2-f0564f500587" containerName="registry"
Jul 24 14:39:59.192549 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.192427    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1f04803-5077-4490-afa2-f0564f500587" containerName="registry"
Jul 24 14:39:59.194978 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.194959    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"
Jul 24 14:39:59.197078 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.197055    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 14:39:59.197078 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.197068    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 14:39:59.205408 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.205383    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"]
Jul 24 14:39:59.224760 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.224720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:59.293326 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.293297    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp4lv\" (UniqueName: \"kubernetes.io/projected/5ea6d492-dc79-444c-a99a-dde655680117-kube-api-access-dp4lv\") pod \"cert-manager-cainjector-559cdd8c87-8slpm\" (UID: \"5ea6d492-dc79-444c-a99a-dde655680117\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"
Jul 24 14:39:59.293432 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.293327    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ea6d492-dc79-444c-a99a-dde655680117-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8slpm\" (UID: \"5ea6d492-dc79-444c-a99a-dde655680117\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"
Jul 24 14:39:59.394609 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.394584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp4lv\" (UniqueName: \"kubernetes.io/projected/5ea6d492-dc79-444c-a99a-dde655680117-kube-api-access-dp4lv\") pod \"cert-manager-cainjector-559cdd8c87-8slpm\" (UID: \"5ea6d492-dc79-444c-a99a-dde655680117\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"
Jul 24 14:39:59.394771 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.394618    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ea6d492-dc79-444c-a99a-dde655680117-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8slpm\" (UID: \"5ea6d492-dc79-444c-a99a-dde655680117\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"
Jul 24 14:39:59.403078 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.403051    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ea6d492-dc79-444c-a99a-dde655680117-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8slpm\" (UID: \"5ea6d492-dc79-444c-a99a-dde655680117\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"
Jul 24 14:39:59.403162 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.403094    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp4lv\" (UniqueName: \"kubernetes.io/projected/5ea6d492-dc79-444c-a99a-dde655680117-kube-api-access-dp4lv\") pod \"cert-manager-cainjector-559cdd8c87-8slpm\" (UID: \"5ea6d492-dc79-444c-a99a-dde655680117\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"
Jul 24 14:39:59.503735 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.503682    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"
Jul 24 14:39:59.626940 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.626907    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8slpm"]
Jul 24 14:39:59.629562 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:39:59.629537    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ea6d492_dc79_444c_a99a_dde655680117.slice/crio-fbbd6a04f414757120cc557786f483692eea155485ac0787c53bafe33da03849 WatchSource:0}: Error finding container fbbd6a04f414757120cc557786f483692eea155485ac0787c53bafe33da03849: Status 404 returned error can't find the container with id fbbd6a04f414757120cc557786f483692eea155485ac0787c53bafe33da03849
Jul 24 14:39:59.631328 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:39:59.631314    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:40:00.581558 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:00.581486    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm" event={"ID":"5ea6d492-dc79-444c-a99a-dde655680117","Type":"ContainerStarted","Data":"fbbd6a04f414757120cc557786f483692eea155485ac0787c53bafe33da03849"}
Jul 24 14:40:02.573997 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:02.573968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:03.574974 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:03.574942    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:03.593028 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:03.593003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm" event={"ID":"5ea6d492-dc79-444c-a99a-dde655680117","Type":"ContainerStarted","Data":"8f0326bea6b7e7bd70d2b16a9cd202bfac799a3a1f5847f99dfa2bd0e7efd59a"}
Jul 24 14:40:03.615719 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:03.615675    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8slpm" podStartSLOduration=1.6852540120000001 podStartE2EDuration="4.615656836s" totalImagesPullingTime="2.930402824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:39:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:40:03.614963583 +0000 UTC m=+388.767964311" watchObservedRunningTime="2026-07-24 14:40:03.615656836 +0000 UTC m=+388.768657562"
Jul 24 14:40:03.683873 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:03.683841    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:07.829318 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:07.829289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:08.568155 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:08.568125    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:08.668513 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:08.668483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:14.387734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:14.387702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:14.488097 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:14.488067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:16.373393 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:16.373360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:30.730049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:30.730015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:30.877534 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:30.877500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:32.509466 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:32.509395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:32.604789 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:32.604765    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:33.115731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.115710    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:33.327164 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.327127    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:33.438803 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.438746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:33.469360 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.469342    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"]
Jul 24 14:40:33.472368 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.472352    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.474805 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.474777    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 14:40:33.474805 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.474791    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 14:40:33.475194 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.475179    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zx88k\""
Jul 24 14:40:33.475330 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.475314    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 14:40:33.475416 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.475347    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 14:40:33.491998 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.491976    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"]
Jul 24 14:40:33.609582 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.609556    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcszm\" (UniqueName: \"kubernetes.io/projected/bc865c7d-dfe1-426b-91ec-f833e581e15c-kube-api-access-hcszm\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.609879 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.609602    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc865c7d-dfe1-426b-91ec-f833e581e15c-webhook-cert\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.609879 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.609636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc865c7d-dfe1-426b-91ec-f833e581e15c-manifests\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.609879 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.609678    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc865c7d-dfe1-426b-91ec-f833e581e15c-tmp\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.609879 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.609721    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc865c7d-dfe1-426b-91ec-f833e581e15c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.625618 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.625589    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:33.710652 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.710598    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc865c7d-dfe1-426b-91ec-f833e581e15c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.710652 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.710630    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcszm\" (UniqueName: \"kubernetes.io/projected/bc865c7d-dfe1-426b-91ec-f833e581e15c-kube-api-access-hcszm\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.710804 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.710656    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc865c7d-dfe1-426b-91ec-f833e581e15c-webhook-cert\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.710804 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.710675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc865c7d-dfe1-426b-91ec-f833e581e15c-manifests\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.710804 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.710707    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc865c7d-dfe1-426b-91ec-f833e581e15c-tmp\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.711107 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.711080    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc865c7d-dfe1-426b-91ec-f833e581e15c-manifests\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.712705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.712681    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc865c7d-dfe1-426b-91ec-f833e581e15c-tmp\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.712819 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.712802    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc865c7d-dfe1-426b-91ec-f833e581e15c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.712971 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.712953    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc865c7d-dfe1-426b-91ec-f833e581e15c-webhook-cert\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.723067 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.723043    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcszm\" (UniqueName: \"kubernetes.io/projected/bc865c7d-dfe1-426b-91ec-f833e581e15c-kube-api-access-hcszm\") pod \"opendatahub-operator-controller-manager-696d5956bc-9m6l4\" (UID: \"bc865c7d-dfe1-426b-91ec-f833e581e15c\") " pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.780026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.780002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:33.781906 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.781888    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:33.873258 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.873224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:33.908536 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.908509    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"]
Jul 24 14:40:33.911800 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:40:33.911770    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc865c7d_dfe1_426b_91ec_f833e581e15c.slice/crio-568b2fb6cf21f047b2274c0529d965a609bd6a55c49f3f863331dd843c30a4ea WatchSource:0}: Error finding container 568b2fb6cf21f047b2274c0529d965a609bd6a55c49f3f863331dd843c30a4ea: Status 404 returned error can't find the container with id 568b2fb6cf21f047b2274c0529d965a609bd6a55c49f3f863331dd843c30a4ea
Jul 24 14:40:33.941408 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:33.941383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:34.034916 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:34.034855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:34.125842 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:34.125824    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:34.683502 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:34.683468    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4" event={"ID":"bc865c7d-dfe1-426b-91ec-f833e581e15c","Type":"ContainerStarted","Data":"568b2fb6cf21f047b2274c0529d965a609bd6a55c49f3f863331dd843c30a4ea"}
Jul 24 14:40:36.691963 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:36.691937    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4" event={"ID":"bc865c7d-dfe1-426b-91ec-f833e581e15c","Type":"ContainerStarted","Data":"7173d9e98057f6aa90cc26656d1e634f7817af62873bc8580ce9a19ef21656da"}
Jul 24 14:40:37.696641 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:37.696617    2583 generic.go:356] "Generic (PLEG): container finished" podID="bc865c7d-dfe1-426b-91ec-f833e581e15c" containerID="7173d9e98057f6aa90cc26656d1e634f7817af62873bc8580ce9a19ef21656da" exitCode=0
Jul 24 14:40:37.696941 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:37.696650    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4" event={"ID":"bc865c7d-dfe1-426b-91ec-f833e581e15c","Type":"ContainerDied","Data":"7173d9e98057f6aa90cc26656d1e634f7817af62873bc8580ce9a19ef21656da"}
Jul 24 14:40:38.701437 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:38.701403    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4" event={"ID":"bc865c7d-dfe1-426b-91ec-f833e581e15c","Type":"ContainerStarted","Data":"9aea7047275c37f8b8ab1d7fa6c74c94bee6792a58d907c1cccb151a7fe4afff"}
Jul 24 14:40:38.701811 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:38.701542    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:38.727498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:38.727433    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4" podStartSLOduration=1.699462412 podStartE2EDuration="5.727418311s" totalImagesPullingTime="3.081267984s" totalInitContainerRuntime="946.687915ms" isStatefulPod=false podCreationTimestamp="2026-07-24 14:40:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:40:38.726930774 +0000 UTC m=+423.879931499" watchObservedRunningTime="2026-07-24 14:40:38.727418311 +0000 UTC m=+423.880419037"
Jul 24 14:40:49.707196 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:49.707166    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-696d5956bc-9m6l4"
Jul 24 14:40:52.570890 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:52.570857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:52.716214 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:52.716182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:58.395475 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.395350    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"]
Jul 24 14:40:58.406749 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.406728    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.409448 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.409401    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 14:40:58.409682 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.409662    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 14:40:58.409815 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.409795    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 14:40:58.412188 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.412173    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 14:40:58.416431 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.416409    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"]
Jul 24 14:40:58.431754 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.431728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:58.568809 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.568785    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e308560-2a6f-41dc-84d6-f83ffaf7bc66-tls-certs\") pod \"kube-auth-proxy-66d67d68d4-r6ngq\" (UID: \"5e308560-2a6f-41dc-84d6-f83ffaf7bc66\") " pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.568919 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.568831    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e308560-2a6f-41dc-84d6-f83ffaf7bc66-tmp\") pod \"kube-auth-proxy-66d67d68d4-r6ngq\" (UID: \"5e308560-2a6f-41dc-84d6-f83ffaf7bc66\") " pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.568919 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.568862    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96pnx\" (UniqueName: \"kubernetes.io/projected/5e308560-2a6f-41dc-84d6-f83ffaf7bc66-kube-api-access-96pnx\") pod \"kube-auth-proxy-66d67d68d4-r6ngq\" (UID: \"5e308560-2a6f-41dc-84d6-f83ffaf7bc66\") " pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.664905 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.664839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:58.669737 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.669716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e308560-2a6f-41dc-84d6-f83ffaf7bc66-tls-certs\") pod \"kube-auth-proxy-66d67d68d4-r6ngq\" (UID: \"5e308560-2a6f-41dc-84d6-f83ffaf7bc66\") " pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.669816 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.669763    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e308560-2a6f-41dc-84d6-f83ffaf7bc66-tmp\") pod \"kube-auth-proxy-66d67d68d4-r6ngq\" (UID: \"5e308560-2a6f-41dc-84d6-f83ffaf7bc66\") " pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.669816 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.669785    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pnx\" (UniqueName: \"kubernetes.io/projected/5e308560-2a6f-41dc-84d6-f83ffaf7bc66-kube-api-access-96pnx\") pod \"kube-auth-proxy-66d67d68d4-r6ngq\" (UID: \"5e308560-2a6f-41dc-84d6-f83ffaf7bc66\") " pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.671998 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.671967    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e308560-2a6f-41dc-84d6-f83ffaf7bc66-tmp\") pod \"kube-auth-proxy-66d67d68d4-r6ngq\" (UID: \"5e308560-2a6f-41dc-84d6-f83ffaf7bc66\") " pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.672190 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.672171    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e308560-2a6f-41dc-84d6-f83ffaf7bc66-tls-certs\") pod \"kube-auth-proxy-66d67d68d4-r6ngq\" (UID: \"5e308560-2a6f-41dc-84d6-f83ffaf7bc66\") " pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.692406 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.692381    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96pnx\" (UniqueName: \"kubernetes.io/projected/5e308560-2a6f-41dc-84d6-f83ffaf7bc66-kube-api-access-96pnx\") pod \"kube-auth-proxy-66d67d68d4-r6ngq\" (UID: \"5e308560-2a6f-41dc-84d6-f83ffaf7bc66\") " pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.714029 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.714002    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2dtr8"]
Jul 24 14:40:58.715935 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.715913    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"
Jul 24 14:40:58.717084 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.717066    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:40:58.720486 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.720465    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 24 14:40:58.728847 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:40:58.728811    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"odh-model-controller-7998c67dc4-2dtr8\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="871167d0-5d94-47d5-8711-52bffbc5c9ae" pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:40:58.728944 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:40:58.728890    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"odh-model-controller-dockercfg-ls49q\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ls49q\"" type="*v1.Secret"
Jul 24 14:40:58.754320 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.754247    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2dtr8"]
Jul 24 14:40:58.838812 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.838769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:58.861383 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.861357    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq"]
Jul 24 14:40:58.863273 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:40:58.863245    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e308560_2a6f_41dc_84d6_f83ffaf7bc66.slice/crio-404a22516d6ca557d32a39fd87bf9915e17a9789bedbcb8f813baa39b92bef80 WatchSource:0}: Error finding container 404a22516d6ca557d32a39fd87bf9915e17a9789bedbcb8f813baa39b92bef80: Status 404 returned error can't find the container with id 404a22516d6ca557d32a39fd87bf9915e17a9789bedbcb8f813baa39b92bef80
Jul 24 14:40:58.871602 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.871576    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwv6x\" (UniqueName: \"kubernetes.io/projected/871167d0-5d94-47d5-8711-52bffbc5c9ae-kube-api-access-rwv6x\") pod \"odh-model-controller-7998c67dc4-2dtr8\" (UID: \"871167d0-5d94-47d5-8711-52bffbc5c9ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:40:58.871718 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.871639    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/871167d0-5d94-47d5-8711-52bffbc5c9ae-cert\") pod \"odh-model-controller-7998c67dc4-2dtr8\" (UID: \"871167d0-5d94-47d5-8711-52bffbc5c9ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:40:58.951474 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.951367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:58.972157 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.972134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/871167d0-5d94-47d5-8711-52bffbc5c9ae-cert\") pod \"odh-model-controller-7998c67dc4-2dtr8\" (UID: \"871167d0-5d94-47d5-8711-52bffbc5c9ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:40:58.972305 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.972185    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwv6x\" (UniqueName: \"kubernetes.io/projected/871167d0-5d94-47d5-8711-52bffbc5c9ae-kube-api-access-rwv6x\") pod \"odh-model-controller-7998c67dc4-2dtr8\" (UID: \"871167d0-5d94-47d5-8711-52bffbc5c9ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:40:58.974693 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.974663    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/871167d0-5d94-47d5-8711-52bffbc5c9ae-cert\") pod \"odh-model-controller-7998c67dc4-2dtr8\" (UID: \"871167d0-5d94-47d5-8711-52bffbc5c9ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:40:58.985228 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:58.985201    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwv6x\" (UniqueName: \"kubernetes.io/projected/871167d0-5d94-47d5-8711-52bffbc5c9ae-kube-api-access-rwv6x\") pod \"odh-model-controller-7998c67dc4-2dtr8\" (UID: \"871167d0-5d94-47d5-8711-52bffbc5c9ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:40:59.577138 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.577103    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ls49q\""
Jul 24 14:40:59.578311 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.578213    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:40:59.631387 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.629976    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hlsvj"]
Jul 24 14:40:59.635650 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.635619    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:40:59.639017 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.638791    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 24 14:40:59.658033 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.657964    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hlsvj"]
Jul 24 14:40:59.691745 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.691680    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:59.745706 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.745612    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2dtr8"]
Jul 24 14:40:59.748996 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:40:59.748969    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871167d0_5d94_47d5_8711_52bffbc5c9ae.slice/crio-4a8c82086315aff769e7a9fa239036ead27c880be5f6f6d08dc5b381a85a012a WatchSource:0}: Error finding container 4a8c82086315aff769e7a9fa239036ead27c880be5f6f6d08dc5b381a85a012a: Status 404 returned error can't find the container with id 4a8c82086315aff769e7a9fa239036ead27c880be5f6f6d08dc5b381a85a012a
Jul 24 14:40:59.768218 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.768186    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8" event={"ID":"871167d0-5d94-47d5-8711-52bffbc5c9ae","Type":"ContainerStarted","Data":"4a8c82086315aff769e7a9fa239036ead27c880be5f6f6d08dc5b381a85a012a"}
Jul 24 14:40:59.769590 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.769563    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq" event={"ID":"5e308560-2a6f-41dc-84d6-f83ffaf7bc66","Type":"ContainerStarted","Data":"404a22516d6ca557d32a39fd87bf9915e17a9789bedbcb8f813baa39b92bef80"}
Jul 24 14:40:59.779394 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.779362    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwxvp\" (UniqueName: \"kubernetes.io/projected/136fc1c8-5b98-4af4-8322-b7fbb686bb39-kube-api-access-bwxvp\") pod \"model-serving-api-54d89bd5fc-hlsvj\" (UID: \"136fc1c8-5b98-4af4-8322-b7fbb686bb39\") " pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:40:59.779514 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.779481    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136fc1c8-5b98-4af4-8322-b7fbb686bb39-tls-certs\") pod \"model-serving-api-54d89bd5fc-hlsvj\" (UID: \"136fc1c8-5b98-4af4-8322-b7fbb686bb39\") " pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:40:59.880618 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.880587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwxvp\" (UniqueName: \"kubernetes.io/projected/136fc1c8-5b98-4af4-8322-b7fbb686bb39-kube-api-access-bwxvp\") pod \"model-serving-api-54d89bd5fc-hlsvj\" (UID: \"136fc1c8-5b98-4af4-8322-b7fbb686bb39\") " pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:40:59.880764 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.880640    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136fc1c8-5b98-4af4-8322-b7fbb686bb39-tls-certs\") pod \"model-serving-api-54d89bd5fc-hlsvj\" (UID: \"136fc1c8-5b98-4af4-8322-b7fbb686bb39\") " pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:40:59.883475 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.883427    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136fc1c8-5b98-4af4-8322-b7fbb686bb39-tls-certs\") pod \"model-serving-api-54d89bd5fc-hlsvj\" (UID: \"136fc1c8-5b98-4af4-8322-b7fbb686bb39\") " pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:40:59.897757 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.897734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:59.898838 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.898816    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwxvp\" (UniqueName: \"kubernetes.io/projected/136fc1c8-5b98-4af4-8322-b7fbb686bb39-kube-api-access-bwxvp\") pod \"model-serving-api-54d89bd5fc-hlsvj\" (UID: \"136fc1c8-5b98-4af4-8322-b7fbb686bb39\") " pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:40:59.951702 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:40:59.951680    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:41:00.111841 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.111811    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hlsvj"]
Jul 24 14:41:00.115114 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:41:00.115083    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136fc1c8_5b98_4af4_8322_b7fbb686bb39.slice/crio-21f98e917fd91a325df6861576fbe19b9e3364a66b2389ee5e2e8cd05fd63a47 WatchSource:0}: Error finding container 21f98e917fd91a325df6861576fbe19b9e3364a66b2389ee5e2e8cd05fd63a47: Status 404 returned error can't find the container with id 21f98e917fd91a325df6861576fbe19b9e3364a66b2389ee5e2e8cd05fd63a47
Jul 24 14:41:00.246892 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.246250    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"]
Jul 24 14:41:00.249671 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.249644    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.252517 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.252492    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 14:41:00.253116 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.253095    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zstr9\""
Jul 24 14:41:00.253214 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.253122    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 14:41:00.253469 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.253426    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 14:41:00.254954 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.254776    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 14:41:00.258079 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.257314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:00.269126 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.269069    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"]
Jul 24 14:41:00.363911 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.363766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:00.385387 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.385354    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.385565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.385400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b5598-b73e-4aa9-a7af-8ff12743270f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.385565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.385430    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.385565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.385512    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.385565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.385563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.385763 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.385595    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grqrw\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-kube-api-access-grqrw\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.385763 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.385625    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.486905 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.486865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.487062 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.486924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grqrw\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-kube-api-access-grqrw\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.487062 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.486958    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.487062 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.487023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.487062 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.487053    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b5598-b73e-4aa9-a7af-8ff12743270f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.487281 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.487076    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.487281 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.487113    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.488187 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.488159    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.490220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.490168    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.490937 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.490911    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b5598-b73e-4aa9-a7af-8ff12743270f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.491427 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.491405    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.493043 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.492934    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.496700 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.496613    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:00.499648 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.499589    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.500673 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.500652    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grqrw\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-kube-api-access-grqrw\") pod \"istiod-openshift-gateway-7b7995d5cd-tbr6c\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.565357 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.565270    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:00.777318 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:00.777230    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj" event={"ID":"136fc1c8-5b98-4af4-8322-b7fbb686bb39","Type":"ContainerStarted","Data":"21f98e917fd91a325df6861576fbe19b9e3364a66b2389ee5e2e8cd05fd63a47"}
Jul 24 14:41:01.597182 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:01.597148    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:01.697944 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:01.697893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:01.966993 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:01.966878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:03.323392 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.323364    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"]
Jul 24 14:41:03.326031 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:41:03.326008    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2b5598_b73e_4aa9_a7af_8ff12743270f.slice/crio-cab3436650044181d971e0903fe5e6595d74cd952330fc708ddfec1c963a0107 WatchSource:0}: Error finding container cab3436650044181d971e0903fe5e6595d74cd952330fc708ddfec1c963a0107: Status 404 returned error can't find the container with id cab3436650044181d971e0903fe5e6595d74cd952330fc708ddfec1c963a0107
Jul 24 14:41:03.787525 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.787487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:03.790818 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.790791    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq" event={"ID":"5e308560-2a6f-41dc-84d6-f83ffaf7bc66","Type":"ContainerStarted","Data":"974bfbd2a074f01caa46386b334778a95ba6d54b5a4efd1e5f3686a0392c3909"}
Jul 24 14:41:03.791996 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.791972    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c" event={"ID":"3d2b5598-b73e-4aa9-a7af-8ff12743270f","Type":"ContainerStarted","Data":"cab3436650044181d971e0903fe5e6595d74cd952330fc708ddfec1c963a0107"}
Jul 24 14:41:03.793160 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.793139    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8" event={"ID":"871167d0-5d94-47d5-8711-52bffbc5c9ae","Type":"ContainerStarted","Data":"4ee75b36c950e866d177aea2485fa101c6d113d605efd07f30546a82f4a104f3"}
Jul 24 14:41:03.793257 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.793190    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:41:03.794382 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.794361    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj" event={"ID":"136fc1c8-5b98-4af4-8322-b7fbb686bb39","Type":"ContainerStarted","Data":"7d49ff3f5b22e12d4807079656fae93467f2ee9f5804c85f3a25b8d97f222b47"}
Jul 24 14:41:03.794486 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.794480    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:41:03.841847 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.841808    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq" podStartSLOduration=1.549641455 podStartE2EDuration="5.841792645s" totalImagesPullingTime="4.29215119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:03.840205798 +0000 UTC m=+448.993206523" watchObservedRunningTime="2026-07-24 14:41:03.841792645 +0000 UTC m=+448.994793372"
Jul 24 14:41:03.863734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.863694    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj" podStartSLOduration=1.8233002790000001 podStartE2EDuration="4.863680063s" totalImagesPullingTime="3.040379784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:03.862761425 +0000 UTC m=+449.015762153" watchObservedRunningTime="2026-07-24 14:41:03.863680063 +0000 UTC m=+449.016680788"
Jul 24 14:41:03.882780 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.882733    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8" podStartSLOduration=2.464608546 podStartE2EDuration="5.882717317s" totalImagesPullingTime="3.418108771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:03.881820403 +0000 UTC m=+449.034821129" watchObservedRunningTime="2026-07-24 14:41:03.882717317 +0000 UTC m=+449.035718042"
Jul 24 14:41:03.888007 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:03.887971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:05.920442 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:05.920406    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:41:05.920826 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:05.920489    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:41:06.693449 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:06.693408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:06.812074 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:06.812038    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c" event={"ID":"3d2b5598-b73e-4aa9-a7af-8ff12743270f","Type":"ContainerStarted","Data":"dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd"}
Jul 24 14:41:06.812254 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:06.812234    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:06.813785 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:06.813763    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-tbr6c container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 14:41:06.813895 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:06.813814    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c" podUID="3d2b5598-b73e-4aa9-a7af-8ff12743270f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 14:41:06.839341 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:06.839301    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c" podStartSLOduration=4.24708322 podStartE2EDuration="6.839289056s" totalImagesPullingTime="2.592205836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:06.837737779 +0000 UTC m=+451.990738494" watchObservedRunningTime="2026-07-24 14:41:06.839289056 +0000 UTC m=+451.992289781"
Jul 24 14:41:07.400306 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.400276    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"]
Jul 24 14:41:07.403304 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.403287    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.409636 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.409610    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rs4mn\""
Jul 24 14:41:07.422913 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.422891    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"]
Jul 24 14:41:07.446466 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.446432    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.446565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.446484    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.446565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.446528    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.446565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.446546    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.446688 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.446598    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c955ff85-db8b-4764-855b-6b0c4730411c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.446688 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.446622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.446688 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.446642    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c955ff85-db8b-4764-855b-6b0c4730411c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.446800 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.446693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c955ff85-db8b-4764-855b-6b0c4730411c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.446800 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.446722    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbbqv\" (UniqueName: \"kubernetes.io/projected/c955ff85-db8b-4764-855b-6b0c4730411c-kube-api-access-tbbqv\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.547926 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.547904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c955ff85-db8b-4764-855b-6b0c4730411c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548053 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.547940    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbbqv\" (UniqueName: \"kubernetes.io/projected/c955ff85-db8b-4764-855b-6b0c4730411c-kube-api-access-tbbqv\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548053 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.547961    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548053 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.547981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548053 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548001    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548053 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548027    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548306 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548142    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c955ff85-db8b-4764-855b-6b0c4730411c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548306 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548189    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548306 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548217    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c955ff85-db8b-4764-855b-6b0c4730411c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548470 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548426    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548521 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548466    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548579 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548538    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548713    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.548899 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.548877    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c955ff85-db8b-4764-855b-6b0c4730411c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.550306 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.550287    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c955ff85-db8b-4764-855b-6b0c4730411c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.550376 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.550350    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c955ff85-db8b-4764-855b-6b0c4730411c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.558121 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.558101    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbbqv\" (UniqueName: \"kubernetes.io/projected/c955ff85-db8b-4764-855b-6b0c4730411c-kube-api-access-tbbqv\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.558203 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.558106    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c955ff85-db8b-4764-855b-6b0c4730411c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9pz22\" (UID: \"c955ff85-db8b-4764-855b-6b0c4730411c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.713438 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.713382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:07.816529 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.816507    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:07.850008 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:07.849976    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"]
Jul 24 14:41:07.856159 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:41:07.856137    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc955ff85_db8b_4764_855b_6b0c4730411c.slice/crio-1ecd894d00610675937994d67036e2d76f422d1b1b09a7fdc7397e37aa678cdc WatchSource:0}: Error finding container 1ecd894d00610675937994d67036e2d76f422d1b1b09a7fdc7397e37aa678cdc: Status 404 returned error can't find the container with id 1ecd894d00610675937994d67036e2d76f422d1b1b09a7fdc7397e37aa678cdc
Jul 24 14:41:08.821035 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:08.821001    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22" event={"ID":"c955ff85-db8b-4764-855b-6b0c4730411c","Type":"ContainerStarted","Data":"1ecd894d00610675937994d67036e2d76f422d1b1b09a7fdc7397e37aa678cdc"}
Jul 24 14:41:09.577796 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:09.577751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:09.718331 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:09.718286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:10.312356 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.312325    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:41:10.312674 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.312389    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:41:10.312674 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.312424    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:41:10.429381 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.429346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:10.640307 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.640278    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"]
Jul 24 14:41:10.643358 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.643340    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:10.646176 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.646156    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-tsv7v\""
Jul 24 14:41:10.648221 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.648202    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 24 14:41:10.651543 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.651522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:10.664730 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.664707    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"]
Jul 24 14:41:10.668500 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.667976    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"]
Jul 24 14:41:10.668500 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.668073    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:10.670729 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.670705    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zlztp\""
Jul 24 14:41:10.672047 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.672026    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 24 14:41:10.672298 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.672275    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjw9v\" (UniqueName: \"kubernetes.io/projected/11622fda-60f5-43aa-b613-ce594d477479-kube-api-access-rjw9v\") pod \"llmisvc-controller-manager-777c6bd7d5-kr4wf\" (UID: \"11622fda-60f5-43aa-b613-ce594d477479\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:10.672382 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.672367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11622fda-60f5-43aa-b613-ce594d477479-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kr4wf\" (UID: \"11622fda-60f5-43aa-b613-ce594d477479\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:10.688765 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.688744    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"]
Jul 24 14:41:10.760489 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.760441    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:10.773327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.773302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11622fda-60f5-43aa-b613-ce594d477479-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kr4wf\" (UID: \"11622fda-60f5-43aa-b613-ce594d477479\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:10.773415 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.773347    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjw9v\" (UniqueName: \"kubernetes.io/projected/11622fda-60f5-43aa-b613-ce594d477479-kube-api-access-rjw9v\") pod \"llmisvc-controller-manager-777c6bd7d5-kr4wf\" (UID: \"11622fda-60f5-43aa-b613-ce594d477479\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:10.773415 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.773373    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tln2c\" (UniqueName: \"kubernetes.io/projected/43fde56f-fe9b-45a9-a5c5-7f7154cb6183-kube-api-access-tln2c\") pod \"kserve-controller-manager-d6f4b65b8-c2lfm\" (UID: \"43fde56f-fe9b-45a9-a5c5-7f7154cb6183\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:10.773415 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.773400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43fde56f-fe9b-45a9-a5c5-7f7154cb6183-cert\") pod \"kserve-controller-manager-d6f4b65b8-c2lfm\" (UID: \"43fde56f-fe9b-45a9-a5c5-7f7154cb6183\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:10.775753 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.775735    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11622fda-60f5-43aa-b613-ce594d477479-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kr4wf\" (UID: \"11622fda-60f5-43aa-b613-ce594d477479\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:10.785426 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.785407    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjw9v\" (UniqueName: \"kubernetes.io/projected/11622fda-60f5-43aa-b613-ce594d477479-kube-api-access-rjw9v\") pod \"llmisvc-controller-manager-777c6bd7d5-kr4wf\" (UID: \"11622fda-60f5-43aa-b613-ce594d477479\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:10.829603 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.829583    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22" event={"ID":"c955ff85-db8b-4764-855b-6b0c4730411c","Type":"ContainerStarted","Data":"34ccd0c39f6f38c7ef9404007db91440dc3b662988fd8ebad1dcc61d0f93f0cd"}
Jul 24 14:41:10.873735 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.873711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tln2c\" (UniqueName: \"kubernetes.io/projected/43fde56f-fe9b-45a9-a5c5-7f7154cb6183-kube-api-access-tln2c\") pod \"kserve-controller-manager-d6f4b65b8-c2lfm\" (UID: \"43fde56f-fe9b-45a9-a5c5-7f7154cb6183\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:10.873989 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.873954    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43fde56f-fe9b-45a9-a5c5-7f7154cb6183-cert\") pod \"kserve-controller-manager-d6f4b65b8-c2lfm\" (UID: \"43fde56f-fe9b-45a9-a5c5-7f7154cb6183\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:10.875984 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.875967    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43fde56f-fe9b-45a9-a5c5-7f7154cb6183-cert\") pod \"kserve-controller-manager-d6f4b65b8-c2lfm\" (UID: \"43fde56f-fe9b-45a9-a5c5-7f7154cb6183\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:10.882026 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.882008    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tln2c\" (UniqueName: \"kubernetes.io/projected/43fde56f-fe9b-45a9-a5c5-7f7154cb6183-kube-api-access-tln2c\") pod \"kserve-controller-manager-d6f4b65b8-c2lfm\" (UID: \"43fde56f-fe9b-45a9-a5c5-7f7154cb6183\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:10.953472 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.953389    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:10.977731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:10.977703    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:11.108427 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.108371    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22" podStartSLOduration=1.6548407680000001 podStartE2EDuration="4.108351167s" totalImagesPullingTime="2.453510399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:10.852976212 +0000 UTC m=+456.005976938" watchObservedRunningTime="2026-07-24 14:41:11.108351167 +0000 UTC m=+456.261351898"
Jul 24 14:41:11.109135 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.109115    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"]
Jul 24 14:41:11.112250 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:41:11.112220    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod11622fda_60f5_43aa_b613_ce594d477479.slice/crio-e52e107f06490b277755707ea6771132e18f637da6d3271704fd56684da3f787 WatchSource:0}: Error finding container e52e107f06490b277755707ea6771132e18f637da6d3271704fd56684da3f787: Status 404 returned error can't find the container with id e52e107f06490b277755707ea6771132e18f637da6d3271704fd56684da3f787
Jul 24 14:41:11.126292 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.126271    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"]
Jul 24 14:41:11.129304 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:41:11.129281    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43fde56f_fe9b_45a9_a5c5_7f7154cb6183.slice/crio-995c179c02bdd6f2c3396c1dbab42371e5eec0c767069a3af5fbab0593a08933 WatchSource:0}: Error finding container 995c179c02bdd6f2c3396c1dbab42371e5eec0c767069a3af5fbab0593a08933: Status 404 returned error can't find the container with id 995c179c02bdd6f2c3396c1dbab42371e5eec0c767069a3af5fbab0593a08933
Jul 24 14:41:11.713795 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.713757    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:11.720334 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.720310    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:11.836945 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.836900    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf" event={"ID":"11622fda-60f5-43aa-b613-ce594d477479","Type":"ContainerStarted","Data":"e52e107f06490b277755707ea6771132e18f637da6d3271704fd56684da3f787"}
Jul 24 14:41:11.838350 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.838301    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm" event={"ID":"43fde56f-fe9b-45a9-a5c5-7f7154cb6183","Type":"ContainerStarted","Data":"995c179c02bdd6f2c3396c1dbab42371e5eec0c767069a3af5fbab0593a08933"}
Jul 24 14:41:11.838720 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.838697    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:11.839888 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.839834    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22"
Jul 24 14:41:11.883644 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.883374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:11.992734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:11.992623    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:12.122537 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:12.122502    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:14.800991 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:14.800962    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2dtr8"
Jul 24 14:41:14.803318 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:14.803296    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hlsvj"
Jul 24 14:41:14.855348 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:14.855216    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm" event={"ID":"43fde56f-fe9b-45a9-a5c5-7f7154cb6183","Type":"ContainerStarted","Data":"ae46e5d381bb8ca207094c9ea2b5bf57ab0cc4fdc0b22fdad7e39e8e87db7d83"}
Jul 24 14:41:14.855348 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:14.855293    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:14.857509 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:14.857481    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf" event={"ID":"11622fda-60f5-43aa-b613-ce594d477479","Type":"ContainerStarted","Data":"7b76a9b77143f3d734d67ee35ad0e26bf249064511700616fedd0e2c03a6541f"}
Jul 24 14:41:14.857669 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:14.857545    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:14.875708 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:14.875651    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm" podStartSLOduration=2.209220533 podStartE2EDuration="4.875625793s" totalImagesPullingTime="2.66640526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:14.87358653 +0000 UTC m=+460.026587257" watchObservedRunningTime="2026-07-24 14:41:14.875625793 +0000 UTC m=+460.028626521"
Jul 24 14:41:14.893353 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:14.893317    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf" podStartSLOduration=2.206278235 podStartE2EDuration="4.893304545s" totalImagesPullingTime="2.68702631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:14.891520099 +0000 UTC m=+460.044520825" watchObservedRunningTime="2026-07-24 14:41:14.893304545 +0000 UTC m=+460.046305267"
Jul 24 14:41:16.847681 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:16.847644    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"]
Jul 24 14:41:16.848105 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:16.847845    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c" podUID="3d2b5598-b73e-4aa9-a7af-8ff12743270f" containerName="discovery" containerID="cri-o://dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd" gracePeriod=30
Jul 24 14:41:16.944360 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:16.944330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:17.084446 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.084424    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:17.122100 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.122050    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-cacerts\") pod \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") "
Jul 24 14:41:17.122100 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.122076    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-kubeconfig\") pod \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") "
Jul 24 14:41:17.122220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.122103    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-ca-configmap\") pod \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") "
Jul 24 14:41:17.122220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.122126    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b5598-b73e-4aa9-a7af-8ff12743270f-local-certs\") pod \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") "
Jul 24 14:41:17.122220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.122165    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-token\") pod \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") "
Jul 24 14:41:17.122220 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.122190    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grqrw\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-kube-api-access-grqrw\") pod \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") "
Jul 24 14:41:17.122408 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.122228    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-dns-cert\") pod \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\" (UID: \"3d2b5598-b73e-4aa9-a7af-8ff12743270f\") "
Jul 24 14:41:17.122549 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.122522    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3d2b5598-b73e-4aa9-a7af-8ff12743270f" (UID: "3d2b5598-b73e-4aa9-a7af-8ff12743270f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:41:17.124624 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.124591    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d2b5598-b73e-4aa9-a7af-8ff12743270f" (UID: "3d2b5598-b73e-4aa9-a7af-8ff12743270f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:41:17.124747 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.124717    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-cacerts" (OuterVolumeSpecName: "cacerts") pod "3d2b5598-b73e-4aa9-a7af-8ff12743270f" (UID: "3d2b5598-b73e-4aa9-a7af-8ff12743270f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:41:17.124747 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.124727    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2b5598-b73e-4aa9-a7af-8ff12743270f-local-certs" (OuterVolumeSpecName: "local-certs") pod "3d2b5598-b73e-4aa9-a7af-8ff12743270f" (UID: "3d2b5598-b73e-4aa9-a7af-8ff12743270f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:41:17.124911 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.124803    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-kube-api-access-grqrw" (OuterVolumeSpecName: "kube-api-access-grqrw") pod "3d2b5598-b73e-4aa9-a7af-8ff12743270f" (UID: "3d2b5598-b73e-4aa9-a7af-8ff12743270f"). InnerVolumeSpecName "kube-api-access-grqrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:41:17.124911 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.124823    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3d2b5598-b73e-4aa9-a7af-8ff12743270f" (UID: "3d2b5598-b73e-4aa9-a7af-8ff12743270f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:41:17.124911 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.124858    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3d2b5598-b73e-4aa9-a7af-8ff12743270f" (UID: "3d2b5598-b73e-4aa9-a7af-8ff12743270f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:41:17.223414 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.223394    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-dns-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:41:17.223414 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.223415    2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-cacerts\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:41:17.223550 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.223425    2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-kubeconfig\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:41:17.223550 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.223434    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-csr-ca-configmap\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:41:17.223550 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.223443    2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2b5598-b73e-4aa9-a7af-8ff12743270f-local-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:41:17.223550 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.223469    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:41:17.223550 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.223478    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grqrw\" (UniqueName: \"kubernetes.io/projected/3d2b5598-b73e-4aa9-a7af-8ff12743270f-kube-api-access-grqrw\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:41:17.870602 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.870574    2583 generic.go:356] "Generic (PLEG): container finished" podID="3d2b5598-b73e-4aa9-a7af-8ff12743270f" containerID="dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd" exitCode=0
Jul 24 14:41:17.870953 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.870610    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"
Jul 24 14:41:17.870953 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.870644    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c" event={"ID":"3d2b5598-b73e-4aa9-a7af-8ff12743270f","Type":"ContainerDied","Data":"dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd"}
Jul 24 14:41:17.870953 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.870693    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c" event={"ID":"3d2b5598-b73e-4aa9-a7af-8ff12743270f","Type":"ContainerDied","Data":"cab3436650044181d971e0903fe5e6595d74cd952330fc708ddfec1c963a0107"}
Jul 24 14:41:17.870953 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.870724    2583 scope.go:117] "RemoveContainer" containerID="dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd"
Jul 24 14:41:17.878285 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.878265    2583 scope.go:117] "RemoveContainer" containerID="dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd"
Jul 24 14:41:17.878604 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:41:17.878576    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd\": container with ID starting with dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd not found: ID does not exist" containerID="dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd"
Jul 24 14:41:17.878663 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.878616    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd"} err="failed to get container status \"dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd\": rpc error: code = NotFound desc = could not find container \"dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd\": container with ID starting with dfa4f5a2738dd27fa8d744d82298b7ce3d151bc2dd51848ca6219b90a72cc1cd not found: ID does not exist"
Jul 24 14:41:17.888231 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.888210    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"]
Jul 24 14:41:17.893327 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:17.893307    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c"]
Jul 24 14:41:19.341652 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:19.341622    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d2b5598-b73e-4aa9-a7af-8ff12743270f" path="/var/lib/kubelet/pods/3d2b5598-b73e-4aa9-a7af-8ff12743270f/volumes"
Jul 24 14:41:45.864420 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:45.864393    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kr4wf"
Jul 24 14:41:45.867109 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:45.867088    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c2lfm"
Jul 24 14:41:54.090698 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.090664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:54.188718 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.188686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:54.239258 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.239226    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbz8t"]
Jul 24 14:41:54.239578 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.239562    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d2b5598-b73e-4aa9-a7af-8ff12743270f" containerName="discovery"
Jul 24 14:41:54.239670 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.239580    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d2b5598-b73e-4aa9-a7af-8ff12743270f" containerName="discovery"
Jul 24 14:41:54.239670 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.239646    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d2b5598-b73e-4aa9-a7af-8ff12743270f" containerName="discovery"
Jul 24 14:41:54.250680 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.250656    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t"
Jul 24 14:41:54.252470 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.252434    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbz8t"]
Jul 24 14:41:54.253021 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.253001    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6hwkb\""
Jul 24 14:41:54.253131 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.253003    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 14:41:54.253936 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.253918    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 14:41:54.304756 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.304727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:54.370041 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.370018    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msqz5\" (UniqueName: \"kubernetes.io/projected/ea222826-f10a-4fcc-bee7-efac4535644f-kube-api-access-msqz5\") pod \"kuadrant-operator-catalog-dbz8t\" (UID: \"ea222826-f10a-4fcc-bee7-efac4535644f\") " pod="kuadrant-system/kuadrant-operator-catalog-dbz8t"
Jul 24 14:41:54.471300 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.471275    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msqz5\" (UniqueName: \"kubernetes.io/projected/ea222826-f10a-4fcc-bee7-efac4535644f-kube-api-access-msqz5\") pod \"kuadrant-operator-catalog-dbz8t\" (UID: \"ea222826-f10a-4fcc-bee7-efac4535644f\") " pod="kuadrant-system/kuadrant-operator-catalog-dbz8t"
Jul 24 14:41:54.479229 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.479206    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msqz5\" (UniqueName: \"kubernetes.io/projected/ea222826-f10a-4fcc-bee7-efac4535644f-kube-api-access-msqz5\") pod \"kuadrant-operator-catalog-dbz8t\" (UID: \"ea222826-f10a-4fcc-bee7-efac4535644f\") " pod="kuadrant-system/kuadrant-operator-catalog-dbz8t"
Jul 24 14:41:54.560795 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.560779    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t"
Jul 24 14:41:54.593369 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.593342    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbz8t"]
Jul 24 14:41:54.678951 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.678924    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbz8t"]
Jul 24 14:41:54.683125 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:41:54.683102    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea222826_f10a_4fcc_bee7_efac4535644f.slice/crio-69ccfd732f24cd66a338383d969bb8516f2444dbd715c7161a4df5e78eaf191a WatchSource:0}: Error finding container 69ccfd732f24cd66a338383d969bb8516f2444dbd715c7161a4df5e78eaf191a: Status 404 returned error can't find the container with id 69ccfd732f24cd66a338383d969bb8516f2444dbd715c7161a4df5e78eaf191a
Jul 24 14:41:54.804806 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.804782    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rj4tp"]
Jul 24 14:41:54.807339 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.807322    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rj4tp"
Jul 24 14:41:54.816605 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.816578    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rj4tp"]
Jul 24 14:41:54.873738 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.873718    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb4dp\" (UniqueName: \"kubernetes.io/projected/761688fb-2c88-46bb-9dfc-5aca73e07a71-kube-api-access-fb4dp\") pod \"kuadrant-operator-catalog-rj4tp\" (UID: \"761688fb-2c88-46bb-9dfc-5aca73e07a71\") " pod="kuadrant-system/kuadrant-operator-catalog-rj4tp"
Jul 24 14:41:54.974376 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.974332    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb4dp\" (UniqueName: \"kubernetes.io/projected/761688fb-2c88-46bb-9dfc-5aca73e07a71-kube-api-access-fb4dp\") pod \"kuadrant-operator-catalog-rj4tp\" (UID: \"761688fb-2c88-46bb-9dfc-5aca73e07a71\") " pod="kuadrant-system/kuadrant-operator-catalog-rj4tp"
Jul 24 14:41:54.983249 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:54.983224    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb4dp\" (UniqueName: \"kubernetes.io/projected/761688fb-2c88-46bb-9dfc-5aca73e07a71-kube-api-access-fb4dp\") pod \"kuadrant-operator-catalog-rj4tp\" (UID: \"761688fb-2c88-46bb-9dfc-5aca73e07a71\") " pod="kuadrant-system/kuadrant-operator-catalog-rj4tp"
Jul 24 14:41:55.002123 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:55.002101    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t" event={"ID":"ea222826-f10a-4fcc-bee7-efac4535644f","Type":"ContainerStarted","Data":"69ccfd732f24cd66a338383d969bb8516f2444dbd715c7161a4df5e78eaf191a"}
Jul 24 14:41:55.116153 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:55.116127    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rj4tp"
Jul 24 14:41:55.255740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:55.255668    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rj4tp"]
Jul 24 14:41:55.258223 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:41:55.258198    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761688fb_2c88_46bb_9dfc_5aca73e07a71.slice/crio-c2c7747a807a51c2cb557032e55b8d2fe6b5bab28f9d205726d3d71d4ed0932f WatchSource:0}: Error finding container c2c7747a807a51c2cb557032e55b8d2fe6b5bab28f9d205726d3d71d4ed0932f: Status 404 returned error can't find the container with id c2c7747a807a51c2cb557032e55b8d2fe6b5bab28f9d205726d3d71d4ed0932f
Jul 24 14:41:56.010106 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:56.010068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rj4tp" event={"ID":"761688fb-2c88-46bb-9dfc-5aca73e07a71","Type":"ContainerStarted","Data":"c2c7747a807a51c2cb557032e55b8d2fe6b5bab28f9d205726d3d71d4ed0932f"}
Jul 24 14:41:58.019006 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:58.018969    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t" event={"ID":"ea222826-f10a-4fcc-bee7-efac4535644f","Type":"ContainerStarted","Data":"ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6"}
Jul 24 14:41:58.019381 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:58.019010    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t" podUID="ea222826-f10a-4fcc-bee7-efac4535644f" containerName="registry-server" containerID="cri-o://ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6" gracePeriod=2
Jul 24 14:41:58.020434 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:58.020414    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rj4tp" event={"ID":"761688fb-2c88-46bb-9dfc-5aca73e07a71","Type":"ContainerStarted","Data":"1370b1264181190dc4aadf405a11d834f889c3159b988451c90445112bef2736"}
Jul 24 14:41:58.055528 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:58.055486    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t" podStartSLOduration=1.281283933 podStartE2EDuration="4.055474488s" totalImagesPullingTime="2.774190555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:58.037179536 +0000 UTC m=+503.190180263" watchObservedRunningTime="2026-07-24 14:41:58.055474488 +0000 UTC m=+503.208475207"
Jul 24 14:41:58.055984 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:58.055962    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rj4tp" podStartSLOduration=2.286395896 podStartE2EDuration="4.055956447s" totalImagesPullingTime="1.769560551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:58.05528456 +0000 UTC m=+503.208285286" watchObservedRunningTime="2026-07-24 14:41:58.055956447 +0000 UTC m=+503.208957172"
Jul 24 14:41:58.256464 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:58.256425    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t"
Jul 24 14:41:58.404665 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:58.404636    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msqz5\" (UniqueName: \"kubernetes.io/projected/ea222826-f10a-4fcc-bee7-efac4535644f-kube-api-access-msqz5\") pod \"ea222826-f10a-4fcc-bee7-efac4535644f\" (UID: \"ea222826-f10a-4fcc-bee7-efac4535644f\") "
Jul 24 14:41:58.406619 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:58.406582    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea222826-f10a-4fcc-bee7-efac4535644f-kube-api-access-msqz5" (OuterVolumeSpecName: "kube-api-access-msqz5") pod "ea222826-f10a-4fcc-bee7-efac4535644f" (UID: "ea222826-f10a-4fcc-bee7-efac4535644f"). InnerVolumeSpecName "kube-api-access-msqz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:41:58.505594 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:58.505564    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msqz5\" (UniqueName: \"kubernetes.io/projected/ea222826-f10a-4fcc-bee7-efac4535644f-kube-api-access-msqz5\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:41:59.026127 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.026103    2583 generic.go:356] "Generic (PLEG): container finished" podID="ea222826-f10a-4fcc-bee7-efac4535644f" containerID="ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6" exitCode=0
Jul 24 14:41:59.026501 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.026145    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t"
Jul 24 14:41:59.026501 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.026185    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t" event={"ID":"ea222826-f10a-4fcc-bee7-efac4535644f","Type":"ContainerDied","Data":"ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6"}
Jul 24 14:41:59.026501 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.026230    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dbz8t" event={"ID":"ea222826-f10a-4fcc-bee7-efac4535644f","Type":"ContainerDied","Data":"69ccfd732f24cd66a338383d969bb8516f2444dbd715c7161a4df5e78eaf191a"}
Jul 24 14:41:59.026501 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.026249    2583 scope.go:117] "RemoveContainer" containerID="ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6"
Jul 24 14:41:59.036225 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.036204    2583 scope.go:117] "RemoveContainer" containerID="ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6"
Jul 24 14:41:59.036655 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:41:59.036629    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6\": container with ID starting with ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6 not found: ID does not exist" containerID="ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6"
Jul 24 14:41:59.036745 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.036656    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6"} err="failed to get container status \"ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6\": rpc error: code = NotFound desc = could not find container \"ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6\": container with ID starting with ebbc8da7724e1b11875ae184bfb2a8c622ff44ac053915517b0e5ded35a5eca6 not found: ID does not exist"
Jul 24 14:41:59.059996 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.059974    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbz8t"]
Jul 24 14:41:59.065153 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.065134    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbz8t"]
Jul 24 14:41:59.342105 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:41:59.342055    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea222826-f10a-4fcc-bee7-efac4535644f" path="/var/lib/kubelet/pods/ea222826-f10a-4fcc-bee7-efac4535644f/volumes"
Jul 24 14:42:05.116759 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:05.116727    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-rj4tp"
Jul 24 14:42:05.117106 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:05.116783    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-rj4tp"
Jul 24 14:42:05.136945 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:05.136920    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-rj4tp"
Jul 24 14:42:06.068874 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:06.068850    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-rj4tp"
Jul 24 14:42:18.014376 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:18.014336    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:20.026481 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.026438    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"]
Jul 24 14:42:20.026811 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.026716    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea222826-f10a-4fcc-bee7-efac4535644f" containerName="registry-server"
Jul 24 14:42:20.026811 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.026726    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea222826-f10a-4fcc-bee7-efac4535644f" containerName="registry-server"
Jul 24 14:42:20.026811 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.026776    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea222826-f10a-4fcc-bee7-efac4535644f" containerName="registry-server"
Jul 24 14:42:20.030389 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.030369    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:20.032680 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.032657    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-n72xh\""
Jul 24 14:42:20.043831 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.043811    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"]
Jul 24 14:42:20.149866 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.149844    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx5g2\" (UniqueName: \"kubernetes.io/projected/7120bcff-a0e3-4c45-a9d6-4d0d51343e67-kube-api-access-jx5g2\") pod \"limitador-operator-controller-manager-859d7b78bd-llrgl\" (UID: \"7120bcff-a0e3-4c45-a9d6-4d0d51343e67\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:20.250979 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.250956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx5g2\" (UniqueName: \"kubernetes.io/projected/7120bcff-a0e3-4c45-a9d6-4d0d51343e67-kube-api-access-jx5g2\") pod \"limitador-operator-controller-manager-859d7b78bd-llrgl\" (UID: \"7120bcff-a0e3-4c45-a9d6-4d0d51343e67\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:20.265562 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.265543    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx5g2\" (UniqueName: \"kubernetes.io/projected/7120bcff-a0e3-4c45-a9d6-4d0d51343e67-kube-api-access-jx5g2\") pod \"limitador-operator-controller-manager-859d7b78bd-llrgl\" (UID: \"7120bcff-a0e3-4c45-a9d6-4d0d51343e67\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:20.339635 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.339580    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:20.461848 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:20.461825    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"]
Jul 24 14:42:20.464327 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:42:20.464299    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7120bcff_a0e3_4c45_a9d6_4d0d51343e67.slice/crio-6f137eb2474f5cc4a34ddc172b26c3e44c8841bba5dfc999bbf4a13dde30d58a WatchSource:0}: Error finding container 6f137eb2474f5cc4a34ddc172b26c3e44c8841bba5dfc999bbf4a13dde30d58a: Status 404 returned error can't find the container with id 6f137eb2474f5cc4a34ddc172b26c3e44c8841bba5dfc999bbf4a13dde30d58a
Jul 24 14:42:21.110034 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:21.109949    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl" event={"ID":"7120bcff-a0e3-4c45-a9d6-4d0d51343e67","Type":"ContainerStarted","Data":"6f137eb2474f5cc4a34ddc172b26c3e44c8841bba5dfc999bbf4a13dde30d58a"}
Jul 24 14:42:21.647889 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:21.647824    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:22.115772 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:22.115745    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl" event={"ID":"7120bcff-a0e3-4c45-a9d6-4d0d51343e67","Type":"ContainerStarted","Data":"5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6"}
Jul 24 14:42:22.116070 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:22.115844    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:22.137431 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:22.137391    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl" podStartSLOduration=0.893692477 podStartE2EDuration="2.137378227s" totalImagesPullingTime="1.24368575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:42:22.135699243 +0000 UTC m=+527.288699969" watchObservedRunningTime="2026-07-24 14:42:22.137378227 +0000 UTC m=+527.290378951"
Jul 24 14:42:23.805094 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:23.805059    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"]
Jul 24 14:42:23.808305 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:23.808289    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"
Jul 24 14:42:23.810489 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:23.810444    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zcvxd\""
Jul 24 14:42:23.810589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:23.810464    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 24 14:42:23.822132 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:23.822110    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"]
Jul 24 14:42:23.975395 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:23.975367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t942j\" (UniqueName: \"kubernetes.io/projected/ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c-kube-api-access-t942j\") pod \"dns-operator-controller-manager-864dd78c4c-hxgvq\" (UID: \"ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"
Jul 24 14:42:24.076680 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:24.076628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t942j\" (UniqueName: \"kubernetes.io/projected/ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c-kube-api-access-t942j\") pod \"dns-operator-controller-manager-864dd78c4c-hxgvq\" (UID: \"ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"
Jul 24 14:42:24.085962 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:24.085933    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t942j\" (UniqueName: \"kubernetes.io/projected/ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c-kube-api-access-t942j\") pod \"dns-operator-controller-manager-864dd78c4c-hxgvq\" (UID: \"ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"
Jul 24 14:42:24.118494 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:24.118472    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"
Jul 24 14:42:24.247882 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:24.247849    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"]
Jul 24 14:42:24.251847 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:42:24.251819    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6d4a4b_1ff4_4f95_8ca9_658e5c5f963c.slice/crio-85c352e59a0462ec29d6dfdb25e12aa39398f43680117e8622cfc8a51cd9363d WatchSource:0}: Error finding container 85c352e59a0462ec29d6dfdb25e12aa39398f43680117e8622cfc8a51cd9363d: Status 404 returned error can't find the container with id 85c352e59a0462ec29d6dfdb25e12aa39398f43680117e8622cfc8a51cd9363d
Jul 24 14:42:24.386692 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:24.386668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:25.132411 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:25.132369    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq" event={"ID":"ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c","Type":"ContainerStarted","Data":"85c352e59a0462ec29d6dfdb25e12aa39398f43680117e8622cfc8a51cd9363d"}
Jul 24 14:42:26.137828 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:26.137801    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq" event={"ID":"ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c","Type":"ContainerStarted","Data":"4b5428e10560aae6fa18fa0823fcba4904a2cea8f0437a6c8bf48d886279e80f"}
Jul 24 14:42:26.138146 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:26.137898    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"
Jul 24 14:42:26.178609 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:26.178564    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq" podStartSLOduration=1.664854 podStartE2EDuration="3.17855129s" totalImagesPullingTime="1.51369729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:42:26.177329393 +0000 UTC m=+531.330330117" watchObservedRunningTime="2026-07-24 14:42:26.17855129 +0000 UTC m=+531.331552015"
Jul 24 14:42:26.441228 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:26.441150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:27.663408 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:27.663376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:28.916777 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:28.916744    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"]
Jul 24 14:42:28.924964 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:28.924945    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:28.927247 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:28.927225    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-knv64\""
Jul 24 14:42:28.943048 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:28.943025    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"]
Jul 24 14:42:29.012895 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.012868    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/379ae114-6ff1-4ccd-91d9-a3299b463530-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" (UID: \"379ae114-6ff1-4ccd-91d9-a3299b463530\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:29.012986 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.012904    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdqwk\" (UniqueName: \"kubernetes.io/projected/379ae114-6ff1-4ccd-91d9-a3299b463530-kube-api-access-wdqwk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" (UID: \"379ae114-6ff1-4ccd-91d9-a3299b463530\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:29.113432 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.113409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/379ae114-6ff1-4ccd-91d9-a3299b463530-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" (UID: \"379ae114-6ff1-4ccd-91d9-a3299b463530\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:29.113532 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.113443    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdqwk\" (UniqueName: \"kubernetes.io/projected/379ae114-6ff1-4ccd-91d9-a3299b463530-kube-api-access-wdqwk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" (UID: \"379ae114-6ff1-4ccd-91d9-a3299b463530\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:29.113842 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.113826    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/379ae114-6ff1-4ccd-91d9-a3299b463530-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" (UID: \"379ae114-6ff1-4ccd-91d9-a3299b463530\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:29.125537 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.125507    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdqwk\" (UniqueName: \"kubernetes.io/projected/379ae114-6ff1-4ccd-91d9-a3299b463530-kube-api-access-wdqwk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" (UID: \"379ae114-6ff1-4ccd-91d9-a3299b463530\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:29.172015 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.171959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:29.234774 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.234755    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:29.364591 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.364559    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"]
Jul 24 14:42:29.368722 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:42:29.368690    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379ae114_6ff1_4ccd_91d9_a3299b463530.slice/crio-99dc1518cd8794d568cc9f8de5e39a78c9e1abd5a66b97ea2af1c015a756c6d3 WatchSource:0}: Error finding container 99dc1518cd8794d568cc9f8de5e39a78c9e1abd5a66b97ea2af1c015a756c6d3: Status 404 returned error can't find the container with id 99dc1518cd8794d568cc9f8de5e39a78c9e1abd5a66b97ea2af1c015a756c6d3
Jul 24 14:42:29.782635 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:29.782604    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:30.155651 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:30.155607    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx" event={"ID":"379ae114-6ff1-4ccd-91d9-a3299b463530","Type":"ContainerStarted","Data":"99dc1518cd8794d568cc9f8de5e39a78c9e1abd5a66b97ea2af1c015a756c6d3"}
Jul 24 14:42:32.033471 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.033422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:32.128131 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.128096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:32.594816 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.594751    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-drndx"]
Jul 24 14:42:32.598617 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.598594    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx"
Jul 24 14:42:32.602061 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.602039    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-mkwjg\""
Jul 24 14:42:32.609502 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.609437    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-drndx"]
Jul 24 14:42:32.742248 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.742214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grlsn\" (UniqueName: \"kubernetes.io/projected/4b20f872-19aa-414d-98ff-f907c5cbeba0-kube-api-access-grlsn\") pod \"authorino-operator-75cc8d5cb7-drndx\" (UID: \"4b20f872-19aa-414d-98ff-f907c5cbeba0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx"
Jul 24 14:42:32.843169 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.843128    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grlsn\" (UniqueName: \"kubernetes.io/projected/4b20f872-19aa-414d-98ff-f907c5cbeba0-kube-api-access-grlsn\") pod \"authorino-operator-75cc8d5cb7-drndx\" (UID: \"4b20f872-19aa-414d-98ff-f907c5cbeba0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx"
Jul 24 14:42:32.854214 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.854141    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grlsn\" (UniqueName: \"kubernetes.io/projected/4b20f872-19aa-414d-98ff-f907c5cbeba0-kube-api-access-grlsn\") pod \"authorino-operator-75cc8d5cb7-drndx\" (UID: \"4b20f872-19aa-414d-98ff-f907c5cbeba0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx"
Jul 24 14:42:32.910930 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:32.910901    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx"
Jul 24 14:42:33.122165 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:33.122095    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:33.553497 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:33.553442    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-drndx"]
Jul 24 14:42:33.556187 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:42:33.556157    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b20f872_19aa_414d_98ff_f907c5cbeba0.slice/crio-2a38bcdaf9bfe0c9d2654972e401721cfa933b84a067540256fe298dc1e54419 WatchSource:0}: Error finding container 2a38bcdaf9bfe0c9d2654972e401721cfa933b84a067540256fe298dc1e54419: Status 404 returned error can't find the container with id 2a38bcdaf9bfe0c9d2654972e401721cfa933b84a067540256fe298dc1e54419
Jul 24 14:42:34.175798 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:34.175759    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx" event={"ID":"379ae114-6ff1-4ccd-91d9-a3299b463530","Type":"ContainerStarted","Data":"2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d"}
Jul 24 14:42:34.176258 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:34.175915    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:34.177235 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:34.177201    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx" event={"ID":"4b20f872-19aa-414d-98ff-f907c5cbeba0","Type":"ContainerStarted","Data":"2a38bcdaf9bfe0c9d2654972e401721cfa933b84a067540256fe298dc1e54419"}
Jul 24 14:42:34.201216 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:34.201163    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx" podStartSLOduration=2.109960448 podStartE2EDuration="6.201142561s" totalImagesPullingTime="4.091182113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:42:34.198966515 +0000 UTC m=+539.351967241" watchObservedRunningTime="2026-07-24 14:42:34.201142561 +0000 UTC m=+539.354143288"
Jul 24 14:42:35.250768 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:35.250734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:36.186579 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:36.186547    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx" event={"ID":"4b20f872-19aa-414d-98ff-f907c5cbeba0","Type":"ContainerStarted","Data":"aa7bfefefe08a1778c45aaeead8cfcb1d8495799d48dbf399f8e69d2000bf2a7"}
Jul 24 14:42:36.186769 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:36.186612    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx"
Jul 24 14:42:36.205901 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:36.205854    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx" podStartSLOduration=2.150518873 podStartE2EDuration="4.205840859s" totalImagesPullingTime="2.055321986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:42:36.205177683 +0000 UTC m=+541.358178411" watchObservedRunningTime="2026-07-24 14:42:36.205840859 +0000 UTC m=+541.358841584"
Jul 24 14:42:36.291081 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:36.291051    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:36.426529 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:36.426496    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:37.144565 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:37.144544    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hxgvq"
Jul 24 14:42:37.400163 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:37.400087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:37.589676 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:37.589645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:42.835499 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:42.835468    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:42.908262 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:42.908236    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:45.183951 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:45.183924    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:46.174805 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.174776    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"]
Jul 24 14:42:46.178173 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.178157    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:46.191441 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.191417    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"]
Jul 24 14:42:46.237734 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.237708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e2988b-fa97-4ce1-acc4-5d04e248ef66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" (UID: \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:46.237839 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.237737    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2dxq\" (UniqueName: \"kubernetes.io/projected/33e2988b-fa97-4ce1-acc4-5d04e248ef66-kube-api-access-x2dxq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" (UID: \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:46.338446 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.338421    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e2988b-fa97-4ce1-acc4-5d04e248ef66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" (UID: \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:46.338555 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.338469    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2dxq\" (UniqueName: \"kubernetes.io/projected/33e2988b-fa97-4ce1-acc4-5d04e248ef66-kube-api-access-x2dxq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" (UID: \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:46.338759 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.338741    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e2988b-fa97-4ce1-acc4-5d04e248ef66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" (UID: \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:46.347036 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.347014    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2dxq\" (UniqueName: \"kubernetes.io/projected/33e2988b-fa97-4ce1-acc4-5d04e248ef66-kube-api-access-x2dxq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" (UID: \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:46.488212 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.488164    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:46.617244 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.617215    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"]
Jul 24 14:42:46.621130 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:42:46.621102    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e2988b_fa97_4ce1_acc4_5d04e248ef66.slice/crio-020e203f393a4db96d00970b68f71d992be0fb8ab43e3d547c3546138a0c159b WatchSource:0}: Error finding container 020e203f393a4db96d00970b68f71d992be0fb8ab43e3d547c3546138a0c159b: Status 404 returned error can't find the container with id 020e203f393a4db96d00970b68f71d992be0fb8ab43e3d547c3546138a0c159b
Jul 24 14:42:46.898486 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.898449    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"]
Jul 24 14:42:46.902641 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.902623    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"]
Jul 24 14:42:46.916003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.915981    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"]
Jul 24 14:42:46.916227 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.916203    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" containerName="manager" containerID="cri-o://2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d" gracePeriod=2
Jul 24 14:42:46.929069 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.929044    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"]
Jul 24 14:42:46.929595 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.929573    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" containerName="manager"
Jul 24 14:42:46.929703 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.929598    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" containerName="manager"
Jul 24 14:42:46.929703 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.929697    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" containerName="manager"
Jul 24 14:42:46.932665 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.932646    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"]
Jul 24 14:42:46.932792 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.932751    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:46.937431 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:46.937291    2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"379ae114-6ff1-4ccd-91d9-a3299b463530\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-fxprx\": pods \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:46.940303 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:46.940275    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:46.942152 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.942132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fffb74a-93ea-4cde-8625-df141e0c3bd7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6vzkc\" (UID: \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:46.942267 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.942187    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qdb2\" (UniqueName: \"kubernetes.io/projected/7fffb74a-93ea-4cde-8625-df141e0c3bd7-kube-api-access-5qdb2\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6vzkc\" (UID: \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:46.947443 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.947424    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"]
Jul 24 14:42:46.948199 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.947695    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" containerName="manager" containerID="cri-o://5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6" gracePeriod=2
Jul 24 14:42:46.949472 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.949401    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"]
Jul 24 14:42:46.950726 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.949871    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" containerName="manager"
Jul 24 14:42:46.950726 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.949891    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" containerName="manager"
Jul 24 14:42:46.950726 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.949985    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" containerName="manager"
Jul 24 14:42:46.953216 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.953025    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"]
Jul 24 14:42:46.953216 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.953116    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"]
Jul 24 14:42:46.953216 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.953125    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:46.964707 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:46.964685    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"]
Jul 24 14:42:46.996473 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:46.996430    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-llrgl\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:46.998001 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:46.997978    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:47.042628 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.042606    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-djmvm\" (UID: \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:47.042736 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.042650    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fffb74a-93ea-4cde-8625-df141e0c3bd7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6vzkc\" (UID: \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:47.042736 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.042697    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjt4f\" (UniqueName: \"kubernetes.io/projected/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-kube-api-access-wjt4f\") pod \"kuadrant-operator-controller-manager-8684c9ff49-djmvm\" (UID: \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:47.042736 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.042725    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qdb2\" (UniqueName: \"kubernetes.io/projected/7fffb74a-93ea-4cde-8625-df141e0c3bd7-kube-api-access-5qdb2\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6vzkc\" (UID: \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:47.043182 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.043081    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fffb74a-93ea-4cde-8625-df141e0c3bd7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6vzkc\" (UID: \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:47.052213 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.052193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qdb2\" (UniqueName: \"kubernetes.io/projected/7fffb74a-93ea-4cde-8625-df141e0c3bd7-kube-api-access-5qdb2\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-6vzkc\" (UID: \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:47.054405 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.054388    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:47.143999 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.143382    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-djmvm\" (UID: \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:47.143999 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.143490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjt4f\" (UniqueName: \"kubernetes.io/projected/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-kube-api-access-wjt4f\") pod \"kuadrant-operator-controller-manager-8684c9ff49-djmvm\" (UID: \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:47.143999 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.143905    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-djmvm\" (UID: \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:47.153925 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.153862    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjt4f\" (UniqueName: \"kubernetes.io/projected/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-kube-api-access-wjt4f\") pod \"kuadrant-operator-controller-manager-8684c9ff49-djmvm\" (UID: \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:47.183147 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.183128    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:47.185414 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.185387    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-llrgl\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:47.187506 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.187442    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:47.189585 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.189570    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:47.191578 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.191556    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:47.193360 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.193340    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-llrgl\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:47.196950 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.196930    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-drndx"
Jul 24 14:42:47.198849 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.198816    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-llrgl\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:47.200683 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.200663    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:47.237754 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.237680    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg" container="manager"
Jul 24 14:42:47.239248 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.239226    2583 generic.go:356] "Generic (PLEG): container finished" podID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" containerID="5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6" exitCode=0
Jul 24 14:42:47.239357 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.239250    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:47.239357 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.239322    2583 scope.go:117] "RemoveContainer" containerID="5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6"
Jul 24 14:42:47.240862 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.240839    2583 generic.go:356] "Generic (PLEG): container finished" podID="379ae114-6ff1-4ccd-91d9-a3299b463530" containerID="2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d" exitCode=0
Jul 24 14:42:47.240862 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.240874    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:47.243531 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.243504    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fxprx\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxprx"
Jul 24 14:42:47.243834 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.243815    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdqwk\" (UniqueName: \"kubernetes.io/projected/379ae114-6ff1-4ccd-91d9-a3299b463530-kube-api-access-wdqwk\") pod \"379ae114-6ff1-4ccd-91d9-a3299b463530\" (UID: \"379ae114-6ff1-4ccd-91d9-a3299b463530\") "
Jul 24 14:42:47.243918 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.243857    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jx5g2\" (UniqueName: \"kubernetes.io/projected/7120bcff-a0e3-4c45-a9d6-4d0d51343e67-kube-api-access-jx5g2\") pod \"7120bcff-a0e3-4c45-a9d6-4d0d51343e67\" (UID: \"7120bcff-a0e3-4c45-a9d6-4d0d51343e67\") "
Jul 24 14:42:47.243918 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.243906    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/379ae114-6ff1-4ccd-91d9-a3299b463530-extensions-socket-volume\") pod \"379ae114-6ff1-4ccd-91d9-a3299b463530\" (UID: \"379ae114-6ff1-4ccd-91d9-a3299b463530\") "
Jul 24 14:42:47.244423 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.244395    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379ae114-6ff1-4ccd-91d9-a3299b463530-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "379ae114-6ff1-4ccd-91d9-a3299b463530" (UID: "379ae114-6ff1-4ccd-91d9-a3299b463530"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:42:47.245371 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.245346    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:47.245933 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.245907    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/379ae114-6ff1-4ccd-91d9-a3299b463530-kube-api-access-wdqwk" (OuterVolumeSpecName: "kube-api-access-wdqwk") pod "379ae114-6ff1-4ccd-91d9-a3299b463530" (UID: "379ae114-6ff1-4ccd-91d9-a3299b463530"). InnerVolumeSpecName "kube-api-access-wdqwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:42:47.245993 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.245917    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7120bcff-a0e3-4c45-a9d6-4d0d51343e67-kube-api-access-jx5g2" (OuterVolumeSpecName: "kube-api-access-jx5g2") pod "7120bcff-a0e3-4c45-a9d6-4d0d51343e67" (UID: "7120bcff-a0e3-4c45-a9d6-4d0d51343e67"). InnerVolumeSpecName "kube-api-access-jx5g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:42:47.247494 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.247468    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-llrgl\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-llrgl"
Jul 24 14:42:47.250295 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.250255    2583 scope.go:117] "RemoveContainer" containerID="5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6"
Jul 24 14:42:47.250711 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.250683    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6\": container with ID starting with 5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6 not found: ID does not exist" containerID="5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6"
Jul 24 14:42:47.250806 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.250717    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6"} err="failed to get container status \"5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6\": rpc error: code = NotFound desc = could not find container \"5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6\": container with ID starting with 5afe4f41d4525ed9fdfa3cc0a840e1a1abb8de7729ef8a352b5df88328b9a1e6 not found: ID does not exist"
Jul 24 14:42:47.250806 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.250739    2583 scope.go:117] "RemoveContainer" containerID="2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d"
Jul 24 14:42:47.260301 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.260197    2583 scope.go:117] "RemoveContainer" containerID="2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d"
Jul 24 14:42:47.260550 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:47.260521    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d\": container with ID starting with 2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d not found: ID does not exist" containerID="2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d"
Jul 24 14:42:47.260623 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.260552    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d"} err="failed to get container status \"2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d\": rpc error: code = NotFound desc = could not find container \"2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d\": container with ID starting with 2cb42e0f4203b8dd1f0d84d26a7a21a5cf6d0f235bae1f4ccb8d8d5c2972e74d not found: ID does not exist"
Jul 24 14:42:47.342136 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.342116    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="379ae114-6ff1-4ccd-91d9-a3299b463530" path="/var/lib/kubelet/pods/379ae114-6ff1-4ccd-91d9-a3299b463530/volumes"
Jul 24 14:42:47.342398 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.342387    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" path="/var/lib/kubelet/pods/7120bcff-a0e3-4c45-a9d6-4d0d51343e67/volumes"
Jul 24 14:42:47.344488 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.344468    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdqwk\" (UniqueName: \"kubernetes.io/projected/379ae114-6ff1-4ccd-91d9-a3299b463530-kube-api-access-wdqwk\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:42:47.344604 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.344490    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx5g2\" (UniqueName: \"kubernetes.io/projected/7120bcff-a0e3-4c45-a9d6-4d0d51343e67-kube-api-access-jx5g2\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:42:47.344604 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.344504    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/379ae114-6ff1-4ccd-91d9-a3299b463530-extensions-socket-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:42:47.362246 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.362227    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:47.401859 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.401837    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"]
Jul 24 14:42:47.404559 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:42:47.404501    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fffb74a_93ea_4cde_8625_df141e0c3bd7.slice/crio-d8299fedfd1178f06cd801877f4599a36cdfb80f950e545add661b912024a13a WatchSource:0}: Error finding container d8299fedfd1178f06cd801877f4599a36cdfb80f950e545add661b912024a13a: Status 404 returned error can't find the container with id d8299fedfd1178f06cd801877f4599a36cdfb80f950e545add661b912024a13a
Jul 24 14:42:47.510603 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:47.510580    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"]
Jul 24 14:42:47.512691 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:42:47.512667    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1329fbc9_b1b5_4b40_8bda_859e702bb0b6.slice/crio-b8d8880b5b763b63e4c8f25e95b4e55c4fdbf230912c638e39f261e385f8e03e WatchSource:0}: Error finding container b8d8880b5b763b63e4c8f25e95b4e55c4fdbf230912c638e39f261e385f8e03e: Status 404 returned error can't find the container with id b8d8880b5b763b63e4c8f25e95b4e55c4fdbf230912c638e39f261e385f8e03e
Jul 24 14:42:48.249432 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.249397    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc" event={"ID":"7fffb74a-93ea-4cde-8625-df141e0c3bd7","Type":"ContainerStarted","Data":"62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec"}
Jul 24 14:42:48.249950 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.249491    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc" event={"ID":"7fffb74a-93ea-4cde-8625-df141e0c3bd7","Type":"ContainerStarted","Data":"d8299fedfd1178f06cd801877f4599a36cdfb80f950e545add661b912024a13a"}
Jul 24 14:42:48.249950 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.249563    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:48.251149 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.251119    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm" event={"ID":"1329fbc9-b1b5-4b40-8bda-859e702bb0b6","Type":"ContainerStarted","Data":"bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9"}
Jul 24 14:42:48.251297 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.251162    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:48.251297 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.251185    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm" event={"ID":"1329fbc9-b1b5-4b40-8bda-859e702bb0b6","Type":"ContainerStarted","Data":"b8d8880b5b763b63e4c8f25e95b4e55c4fdbf230912c638e39f261e385f8e03e"}
Jul 24 14:42:48.252821 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.252769    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg" podUID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" containerName="manager" containerID="cri-o://aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f" gracePeriod=2
Jul 24 14:42:48.284649 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.284607    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc" podStartSLOduration=2.28459353 podStartE2EDuration="2.28459353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:42:48.284319164 +0000 UTC m=+553.437319891" watchObservedRunningTime="2026-07-24 14:42:48.28459353 +0000 UTC m=+553.437594285"
Jul 24 14:42:48.319723 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.319679    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm" podStartSLOduration=2.319666878 podStartE2EDuration="2.319666878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:42:48.317519955 +0000 UTC m=+553.470520681" watchObservedRunningTime="2026-07-24 14:42:48.319666878 +0000 UTC m=+553.472667603"
Jul 24 14:42:48.488728 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.488707    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:48.490779 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:48.490758    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:48.552537 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.552478    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2dxq\" (UniqueName: \"kubernetes.io/projected/33e2988b-fa97-4ce1-acc4-5d04e248ef66-kube-api-access-x2dxq\") pod \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\" (UID: \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\") "
Jul 24 14:42:48.552616 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.552573    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e2988b-fa97-4ce1-acc4-5d04e248ef66-extensions-socket-volume\") pod \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\" (UID: \"33e2988b-fa97-4ce1-acc4-5d04e248ef66\") "
Jul 24 14:42:48.552904 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.552882    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e2988b-fa97-4ce1-acc4-5d04e248ef66-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "33e2988b-fa97-4ce1-acc4-5d04e248ef66" (UID: "33e2988b-fa97-4ce1-acc4-5d04e248ef66"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:42:48.554289 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.554268    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e2988b-fa97-4ce1-acc4-5d04e248ef66-kube-api-access-x2dxq" (OuterVolumeSpecName: "kube-api-access-x2dxq") pod "33e2988b-fa97-4ce1-acc4-5d04e248ef66" (UID: "33e2988b-fa97-4ce1-acc4-5d04e248ef66"). InnerVolumeSpecName "kube-api-access-x2dxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:42:48.653401 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.653378    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2dxq\" (UniqueName: \"kubernetes.io/projected/33e2988b-fa97-4ce1-acc4-5d04e248ef66-kube-api-access-x2dxq\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:42:48.653401 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:48.653398    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e2988b-fa97-4ce1-acc4-5d04e248ef66-extensions-socket-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:42:49.258848 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:49.258824    2583 generic.go:356] "Generic (PLEG): container finished" podID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" containerID="aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f" exitCode=0
Jul 24 14:42:49.259167 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:49.258865    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:49.259167 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:49.258903    2583 scope.go:117] "RemoveContainer" containerID="aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f"
Jul 24 14:42:49.261136 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:49.261112    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:49.267175 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:49.267158    2583 scope.go:117] "RemoveContainer" containerID="aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f"
Jul 24 14:42:49.267431 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:49.267413    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f\": container with ID starting with aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f not found: ID does not exist" containerID="aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f"
Jul 24 14:42:49.267499 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:49.267439    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f"} err="failed to get container status \"aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f\": rpc error: code = NotFound desc = could not find container \"aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f\": container with ID starting with aa5d5669658b14465e6633fb5108bbb5b7c2a87e5ffee637992475f30ec6ce0f not found: ID does not exist"
Jul 24 14:42:49.268405 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:42:49.268384    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-kxflg\" is forbidden: User \"system:node:ip-10-0-141-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-133.ec2.internal' and this object" podUID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kxflg"
Jul 24 14:42:49.342381 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:49.342355    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e2988b-fa97-4ce1-acc4-5d04e248ef66" path="/var/lib/kubelet/pods/33e2988b-fa97-4ce1-acc4-5d04e248ef66/volumes"
Jul 24 14:42:51.828267 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:51.828240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:51.918516 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:51.918488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:52.011289 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:52.011266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:52.105908 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:52.105887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:54.612267 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:54.612229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:54.706264 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:54.706235    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:57.030614 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:57.030579    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:57.105550 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:57.105521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:59.261148 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.261117    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:59.261515 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.261185    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:42:59.336650 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.336620    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"]
Jul 24 14:42:59.337120 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.337089    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm" podUID="1329fbc9-b1b5-4b40-8bda-859e702bb0b6" containerName="manager" containerID="cri-o://bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9" gracePeriod=10
Jul 24 14:42:59.556264 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.556203    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"]
Jul 24 14:42:59.556695 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.556677    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" containerName="manager"
Jul 24 14:42:59.556744 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.556700    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" containerName="manager"
Jul 24 14:42:59.556804 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.556793    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7120bcff-a0e3-4c45-a9d6-4d0d51343e67" containerName="manager"
Jul 24 14:42:59.559794 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.559773    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:42:59.572646 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.572621    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"]
Jul 24 14:42:59.578410 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.578388    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:42:59.626222 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.626197    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjt4f\" (UniqueName: \"kubernetes.io/projected/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-kube-api-access-wjt4f\") pod \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\" (UID: \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\") "
Jul 24 14:42:59.626349 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.626296    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-extensions-socket-volume\") pod \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\" (UID: \"1329fbc9-b1b5-4b40-8bda-859e702bb0b6\") "
Jul 24 14:42:59.626474 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.626432    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/511360df-9654-4ab9-a5ba-70bdb1512d21-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bwb4l\" (UID: \"511360df-9654-4ab9-a5ba-70bdb1512d21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:42:59.626549 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.626508    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44cjb\" (UniqueName: \"kubernetes.io/projected/511360df-9654-4ab9-a5ba-70bdb1512d21-kube-api-access-44cjb\") pod \"kuadrant-operator-controller-manager-86878c769f-bwb4l\" (UID: \"511360df-9654-4ab9-a5ba-70bdb1512d21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:42:59.626746 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.626717    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1329fbc9-b1b5-4b40-8bda-859e702bb0b6" (UID: "1329fbc9-b1b5-4b40-8bda-859e702bb0b6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:42:59.628540 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.628513    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-kube-api-access-wjt4f" (OuterVolumeSpecName: "kube-api-access-wjt4f") pod "1329fbc9-b1b5-4b40-8bda-859e702bb0b6" (UID: "1329fbc9-b1b5-4b40-8bda-859e702bb0b6"). InnerVolumeSpecName "kube-api-access-wjt4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:42:59.727545 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.727520    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/511360df-9654-4ab9-a5ba-70bdb1512d21-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bwb4l\" (UID: \"511360df-9654-4ab9-a5ba-70bdb1512d21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:42:59.727648 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.727555    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44cjb\" (UniqueName: \"kubernetes.io/projected/511360df-9654-4ab9-a5ba-70bdb1512d21-kube-api-access-44cjb\") pod \"kuadrant-operator-controller-manager-86878c769f-bwb4l\" (UID: \"511360df-9654-4ab9-a5ba-70bdb1512d21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:42:59.727648 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.727618    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-extensions-socket-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:42:59.727648 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.727635    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjt4f\" (UniqueName: \"kubernetes.io/projected/1329fbc9-b1b5-4b40-8bda-859e702bb0b6-kube-api-access-wjt4f\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:42:59.727878 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.727855    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/511360df-9654-4ab9-a5ba-70bdb1512d21-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bwb4l\" (UID: \"511360df-9654-4ab9-a5ba-70bdb1512d21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:42:59.751836 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.751811    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44cjb\" (UniqueName: \"kubernetes.io/projected/511360df-9654-4ab9-a5ba-70bdb1512d21-kube-api-access-44cjb\") pod \"kuadrant-operator-controller-manager-86878c769f-bwb4l\" (UID: \"511360df-9654-4ab9-a5ba-70bdb1512d21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:42:59.884524 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:42:59.884502    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:43:00.014994 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.014967    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"]
Jul 24 14:43:00.018028 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:43:00.017998    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511360df_9654_4ab9_a5ba_70bdb1512d21.slice/crio-172f8e4a425de5200c0ed06f818bcf8ac015fd7d0d145e05881d8a3bdb4c2056 WatchSource:0}: Error finding container 172f8e4a425de5200c0ed06f818bcf8ac015fd7d0d145e05881d8a3bdb4c2056: Status 404 returned error can't find the container with id 172f8e4a425de5200c0ed06f818bcf8ac015fd7d0d145e05881d8a3bdb4c2056
Jul 24 14:43:00.308145 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.308071    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l" event={"ID":"511360df-9654-4ab9-a5ba-70bdb1512d21","Type":"ContainerStarted","Data":"38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649"}
Jul 24 14:43:00.308145 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.308125    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l" event={"ID":"511360df-9654-4ab9-a5ba-70bdb1512d21","Type":"ContainerStarted","Data":"172f8e4a425de5200c0ed06f818bcf8ac015fd7d0d145e05881d8a3bdb4c2056"}
Jul 24 14:43:00.308524 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.308155    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:43:00.309416 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.309391    2583 generic.go:356] "Generic (PLEG): container finished" podID="1329fbc9-b1b5-4b40-8bda-859e702bb0b6" containerID="bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9" exitCode=0
Jul 24 14:43:00.309529 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.309431    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"
Jul 24 14:43:00.309529 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.309488    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm" event={"ID":"1329fbc9-b1b5-4b40-8bda-859e702bb0b6","Type":"ContainerDied","Data":"bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9"}
Jul 24 14:43:00.309602 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.309536    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm" event={"ID":"1329fbc9-b1b5-4b40-8bda-859e702bb0b6","Type":"ContainerDied","Data":"b8d8880b5b763b63e4c8f25e95b4e55c4fdbf230912c638e39f261e385f8e03e"}
Jul 24 14:43:00.309602 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.309559    2583 scope.go:117] "RemoveContainer" containerID="bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9"
Jul 24 14:43:00.317942 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.317924    2583 scope.go:117] "RemoveContainer" containerID="bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9"
Jul 24 14:43:00.318177 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:43:00.318160    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9\": container with ID starting with bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9 not found: ID does not exist" containerID="bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9"
Jul 24 14:43:00.318219 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.318185    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9"} err="failed to get container status \"bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9\": rpc error: code = NotFound desc = could not find container \"bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9\": container with ID starting with bfb7a4815d12caa7104e64fed7d42db496f33ab74754e3034dc3fc3bc255c8d9 not found: ID does not exist"
Jul 24 14:43:00.329528 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.329493    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l" podStartSLOduration=1.329481353 podStartE2EDuration="1.329481353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:43:00.328650111 +0000 UTC m=+565.481650837" watchObservedRunningTime="2026-07-24 14:43:00.329481353 +0000 UTC m=+565.482482078"
Jul 24 14:43:00.349091 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.349060    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"]
Jul 24 14:43:00.352107 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.352086    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-djmvm"]
Jul 24 14:43:00.923990 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:00.923948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:01.019488 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:01.019464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:01.341121 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:01.341058    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1329fbc9-b1b5-4b40-8bda-859e702bb0b6" path="/var/lib/kubelet/pods/1329fbc9-b1b5-4b40-8bda-859e702bb0b6/volumes"
Jul 24 14:43:09.786406 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:09.786363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:09.900728 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:09.900696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:11.317483 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.317442    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:43:11.373288 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.373262    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"]
Jul 24 14:43:11.373479 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.373433    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc" podUID="7fffb74a-93ea-4cde-8625-df141e0c3bd7" containerName="manager" containerID="cri-o://62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec" gracePeriod=10
Jul 24 14:43:11.611013 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.610985    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:43:11.708357 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.708330    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fffb74a-93ea-4cde-8625-df141e0c3bd7-extensions-socket-volume\") pod \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\" (UID: \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\") "
Jul 24 14:43:11.708485 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.708393    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qdb2\" (UniqueName: \"kubernetes.io/projected/7fffb74a-93ea-4cde-8625-df141e0c3bd7-kube-api-access-5qdb2\") pod \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\" (UID: \"7fffb74a-93ea-4cde-8625-df141e0c3bd7\") "
Jul 24 14:43:11.708684 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.708664    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fffb74a-93ea-4cde-8625-df141e0c3bd7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7fffb74a-93ea-4cde-8625-df141e0c3bd7" (UID: "7fffb74a-93ea-4cde-8625-df141e0c3bd7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:43:11.710196 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.710173    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fffb74a-93ea-4cde-8625-df141e0c3bd7-kube-api-access-5qdb2" (OuterVolumeSpecName: "kube-api-access-5qdb2") pod "7fffb74a-93ea-4cde-8625-df141e0c3bd7" (UID: "7fffb74a-93ea-4cde-8625-df141e0c3bd7"). InnerVolumeSpecName "kube-api-access-5qdb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:43:11.809194 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.809176    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qdb2\" (UniqueName: \"kubernetes.io/projected/7fffb74a-93ea-4cde-8625-df141e0c3bd7-kube-api-access-5qdb2\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:43:11.809194 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:11.809193    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fffb74a-93ea-4cde-8625-df141e0c3bd7-extensions-socket-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:43:12.364948 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.364921    2583 generic.go:356] "Generic (PLEG): container finished" podID="7fffb74a-93ea-4cde-8625-df141e0c3bd7" containerID="62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec" exitCode=0
Jul 24 14:43:12.365290 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.364959    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"
Jul 24 14:43:12.365290 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.364970    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc" event={"ID":"7fffb74a-93ea-4cde-8625-df141e0c3bd7","Type":"ContainerDied","Data":"62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec"}
Jul 24 14:43:12.365290 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.365018    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc" event={"ID":"7fffb74a-93ea-4cde-8625-df141e0c3bd7","Type":"ContainerDied","Data":"d8299fedfd1178f06cd801877f4599a36cdfb80f950e545add661b912024a13a"}
Jul 24 14:43:12.365290 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.365041    2583 scope.go:117] "RemoveContainer" containerID="62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec"
Jul 24 14:43:12.373564 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.373550    2583 scope.go:117] "RemoveContainer" containerID="62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec"
Jul 24 14:43:12.373827 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:43:12.373809    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec\": container with ID starting with 62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec not found: ID does not exist" containerID="62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec"
Jul 24 14:43:12.373892 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.373834    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec"} err="failed to get container status \"62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec\": rpc error: code = NotFound desc = could not find container \"62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec\": container with ID starting with 62e47d9c1ad6b9c8de5302427e3b50a1f609462db4afe05e4c150f8628fcbcec not found: ID does not exist"
Jul 24 14:43:12.385243 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.385222    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"]
Jul 24 14:43:12.393372 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.393353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:12.393510 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.393495    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-6vzkc"]
Jul 24 14:43:12.494176 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:12.494157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:13.341466 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:13.341436    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fffb74a-93ea-4cde-8625-df141e0c3bd7" path="/var/lib/kubelet/pods/7fffb74a-93ea-4cde-8625-df141e0c3bd7/volumes"
Jul 24 14:43:15.464413 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.464377    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:15.592840 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.592813    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:15.755257 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.755198    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"]
Jul 24 14:43:15.755493 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.755480    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1329fbc9-b1b5-4b40-8bda-859e702bb0b6" containerName="manager"
Jul 24 14:43:15.755540 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.755494    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1329fbc9-b1b5-4b40-8bda-859e702bb0b6" containerName="manager"
Jul 24 14:43:15.755540 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.755512    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fffb74a-93ea-4cde-8625-df141e0c3bd7" containerName="manager"
Jul 24 14:43:15.755540 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.755518    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fffb74a-93ea-4cde-8625-df141e0c3bd7" containerName="manager"
Jul 24 14:43:15.755636 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.755569    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fffb74a-93ea-4cde-8625-df141e0c3bd7" containerName="manager"
Jul 24 14:43:15.755636 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.755579    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1329fbc9-b1b5-4b40-8bda-859e702bb0b6" containerName="manager"
Jul 24 14:43:15.758138 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.758122    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.760618 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.760601    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mj8fl\""
Jul 24 14:43:15.771663 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.771643    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"]
Jul 24 14:43:15.779954 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.779928    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:15.837228 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.837203    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgtcv\" (UniqueName: \"kubernetes.io/projected/040f935e-679e-4600-808b-aa9900fd2944-kube-api-access-hgtcv\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.837364 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.837238    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.837364 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.837256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040f935e-679e-4600-808b-aa9900fd2944-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.837364 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.837274    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040f935e-679e-4600-808b-aa9900fd2944-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.837364 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.837290    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040f935e-679e-4600-808b-aa9900fd2944-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.837364 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.837325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.837364 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.837361    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.837593 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.837382    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.837593 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.837411    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.922670 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.922640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:15.938718 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.938699    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgtcv\" (UniqueName: \"kubernetes.io/projected/040f935e-679e-4600-808b-aa9900fd2944-kube-api-access-hgtcv\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.938827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.938731    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.938827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.938749    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040f935e-679e-4600-808b-aa9900fd2944-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.938827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.938768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040f935e-679e-4600-808b-aa9900fd2944-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.938827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.938787    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040f935e-679e-4600-808b-aa9900fd2944-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.938988 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.938842    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.938988 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.938887    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.938988 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.938919    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.938988 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.938953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.939258 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.939238    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.939343 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.939320    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.939418 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.939400    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.939497 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.939475    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.939562 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.939548    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040f935e-679e-4600-808b-aa9900fd2944-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.940939 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.940923    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040f935e-679e-4600-808b-aa9900fd2944-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.941344 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.941327    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040f935e-679e-4600-808b-aa9900fd2944-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.948293 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.948272    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgtcv\" (UniqueName: \"kubernetes.io/projected/040f935e-679e-4600-808b-aa9900fd2944-kube-api-access-hgtcv\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:15.948834 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:15.948811    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040f935e-679e-4600-808b-aa9900fd2944-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-v8jkd\" (UID: \"040f935e-679e-4600-808b-aa9900fd2944\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:16.067695 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:16.067636    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:16.193700 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:16.193675    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"]
Jul 24 14:43:16.196258 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:43:16.196233    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040f935e_679e_4600_808b_aa9900fd2944.slice/crio-f03df015df9e25c602cfc95dba526594f6017c4b8857cd08b0a3ed93fd6d9f4f WatchSource:0}: Error finding container f03df015df9e25c602cfc95dba526594f6017c4b8857cd08b0a3ed93fd6d9f4f: Status 404 returned error can't find the container with id f03df015df9e25c602cfc95dba526594f6017c4b8857cd08b0a3ed93fd6d9f4f
Jul 24 14:43:16.198211 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:16.198182    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:43:16.198275 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:16.198246    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:43:16.198309 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:16.198273    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:43:16.384533 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:16.384498    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" event={"ID":"040f935e-679e-4600-808b-aa9900fd2944","Type":"ContainerStarted","Data":"19d810872f4186830707e07eaf6af72da292c4e2ed3b719f99b1d62b44d241b8"}
Jul 24 14:43:16.384689 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:16.384548    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" event={"ID":"040f935e-679e-4600-808b-aa9900fd2944","Type":"ContainerStarted","Data":"f03df015df9e25c602cfc95dba526594f6017c4b8857cd08b0a3ed93fd6d9f4f"}
Jul 24 14:43:16.406224 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:16.406180    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" podStartSLOduration=1.406165015 podStartE2EDuration="1.406165015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:43:16.40492815 +0000 UTC m=+581.557928877" watchObservedRunningTime="2026-07-24 14:43:16.406165015 +0000 UTC m=+581.559165741"
Jul 24 14:43:17.067937 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:17.067913    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:17.068987 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:17.068964    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-v8jkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 14:43:17.069088 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:17.069009    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" podUID="040f935e-679e-4600-808b-aa9900fd2944" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 14:43:18.071399 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:18.071376    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:18.393174 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:18.393151    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:18.394035 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:18.394013    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:43:19.185737 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:19.185707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:19.244623 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:19.244589    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:27.759403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:27.759366    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:27.861398 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:27.861374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:29.918278 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:29.918248    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"]
Jul 24 14:43:29.921566 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:29.921549    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:29.924576 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:29.924547    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 14:43:29.924576 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:29.924576    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-djr82\""
Jul 24 14:43:29.939371 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:29.939347    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"]
Jul 24 14:43:29.943608 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:29.943586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:30.010071 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.010043    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"]
Jul 24 14:43:30.032856 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.032826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:30.043770 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.043750    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8cc5444d-e793-427c-af6b-0ce781421418-config-file\") pod \"limitador-limitador-64c94fd64f-hfmrq\" (UID: \"8cc5444d-e793-427c-af6b-0ce781421418\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:30.043864 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.043800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zckh8\" (UniqueName: \"kubernetes.io/projected/8cc5444d-e793-427c-af6b-0ce781421418-kube-api-access-zckh8\") pod \"limitador-limitador-64c94fd64f-hfmrq\" (UID: \"8cc5444d-e793-427c-af6b-0ce781421418\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:30.095191 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.095163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:30.145060 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.145036    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8cc5444d-e793-427c-af6b-0ce781421418-config-file\") pod \"limitador-limitador-64c94fd64f-hfmrq\" (UID: \"8cc5444d-e793-427c-af6b-0ce781421418\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:30.145178 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.145083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zckh8\" (UniqueName: \"kubernetes.io/projected/8cc5444d-e793-427c-af6b-0ce781421418-kube-api-access-zckh8\") pod \"limitador-limitador-64c94fd64f-hfmrq\" (UID: \"8cc5444d-e793-427c-af6b-0ce781421418\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:30.145677 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.145658    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8cc5444d-e793-427c-af6b-0ce781421418-config-file\") pod \"limitador-limitador-64c94fd64f-hfmrq\" (UID: \"8cc5444d-e793-427c-af6b-0ce781421418\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:30.153707 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.153683    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zckh8\" (UniqueName: \"kubernetes.io/projected/8cc5444d-e793-427c-af6b-0ce781421418-kube-api-access-zckh8\") pod \"limitador-limitador-64c94fd64f-hfmrq\" (UID: \"8cc5444d-e793-427c-af6b-0ce781421418\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:30.230928 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.230877    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:30.367091 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.367071    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"]
Jul 24 14:43:30.369490 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:43:30.369444    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc5444d_e793_427c_af6b_0ce781421418.slice/crio-bcc7bbcbf0d6c58339629f758db488a221a9dce6d78f7b9faaa9e276a2bb2cc0 WatchSource:0}: Error finding container bcc7bbcbf0d6c58339629f758db488a221a9dce6d78f7b9faaa9e276a2bb2cc0: Status 404 returned error can't find the container with id bcc7bbcbf0d6c58339629f758db488a221a9dce6d78f7b9faaa9e276a2bb2cc0
Jul 24 14:43:30.442194 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.442169    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq" event={"ID":"8cc5444d-e793-427c-af6b-0ce781421418","Type":"ContainerStarted","Data":"bcc7bbcbf0d6c58339629f758db488a221a9dce6d78f7b9faaa9e276a2bb2cc0"}
Jul 24 14:43:30.793902 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.793867    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hqmqc"]
Jul 24 14:43:30.797084 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.797061    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hqmqc"
Jul 24 14:43:30.799510 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.799485    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hpsrh\""
Jul 24 14:43:30.809538 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.809516    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hqmqc"]
Jul 24 14:43:30.829765 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.829660    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:30.852751 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.852727    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dpbj\" (UniqueName: \"kubernetes.io/projected/1b1c4b34-407a-452b-9892-ccf9e68dd1e1-kube-api-access-8dpbj\") pod \"authorino-58cd955675-hqmqc\" (UID: \"1b1c4b34-407a-452b-9892-ccf9e68dd1e1\") " pod="kuadrant-system/authorino-58cd955675-hqmqc"
Jul 24 14:43:30.901924 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.901897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:30.954350 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.954324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dpbj\" (UniqueName: \"kubernetes.io/projected/1b1c4b34-407a-452b-9892-ccf9e68dd1e1-kube-api-access-8dpbj\") pod \"authorino-58cd955675-hqmqc\" (UID: \"1b1c4b34-407a-452b-9892-ccf9e68dd1e1\") " pod="kuadrant-system/authorino-58cd955675-hqmqc"
Jul 24 14:43:30.967090 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:30.967062    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dpbj\" (UniqueName: \"kubernetes.io/projected/1b1c4b34-407a-452b-9892-ccf9e68dd1e1-kube-api-access-8dpbj\") pod \"authorino-58cd955675-hqmqc\" (UID: \"1b1c4b34-407a-452b-9892-ccf9e68dd1e1\") " pod="kuadrant-system/authorino-58cd955675-hqmqc"
Jul 24 14:43:31.109245 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:31.109218    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hqmqc"
Jul 24 14:43:31.258076 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:31.258050    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hqmqc"]
Jul 24 14:43:31.261319 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:43:31.261288    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b1c4b34_407a_452b_9892_ccf9e68dd1e1.slice/crio-c2a427eac34b643db913ae5d21bf583363eabe542e92e1411d8863d762574cb8 WatchSource:0}: Error finding container c2a427eac34b643db913ae5d21bf583363eabe542e92e1411d8863d762574cb8: Status 404 returned error can't find the container with id c2a427eac34b643db913ae5d21bf583363eabe542e92e1411d8863d762574cb8
Jul 24 14:43:31.449637 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:31.449555    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hqmqc" event={"ID":"1b1c4b34-407a-452b-9892-ccf9e68dd1e1","Type":"ContainerStarted","Data":"c2a427eac34b643db913ae5d21bf583363eabe542e92e1411d8863d762574cb8"}
Jul 24 14:43:33.461150 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:33.461105    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq" event={"ID":"8cc5444d-e793-427c-af6b-0ce781421418","Type":"ContainerStarted","Data":"4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b"}
Jul 24 14:43:33.461564 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:33.461295    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:33.478663 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:33.478586    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq" podStartSLOduration=2.103102032 podStartE2EDuration="4.478572144s" totalImagesPullingTime="2.375470112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:43:33.47760911 +0000 UTC m=+598.630609835" watchObservedRunningTime="2026-07-24 14:43:33.478572144 +0000 UTC m=+598.631572923"
Jul 24 14:43:35.470755 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:35.470728    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hqmqc" event={"ID":"1b1c4b34-407a-452b-9892-ccf9e68dd1e1","Type":"ContainerStarted","Data":"29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5"}
Jul 24 14:43:35.485806 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:35.485761    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hqmqc" podStartSLOduration=1.803199668 podStartE2EDuration="5.485748905s" totalImagesPullingTime="3.682549237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:43:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:43:35.485018169 +0000 UTC m=+600.638018885" watchObservedRunningTime="2026-07-24 14:43:35.485748905 +0000 UTC m=+600.638749629"
Jul 24 14:43:35.581995 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:35.581969    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hqmqc"]
Jul 24 14:43:37.477835 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:37.477799    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hqmqc" podUID="1b1c4b34-407a-452b-9892-ccf9e68dd1e1" containerName="authorino" containerID="cri-o://29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5" gracePeriod=30
Jul 24 14:43:37.723610 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:37.723590    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hqmqc"
Jul 24 14:43:37.812647 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:37.812585    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dpbj\" (UniqueName: \"kubernetes.io/projected/1b1c4b34-407a-452b-9892-ccf9e68dd1e1-kube-api-access-8dpbj\") pod \"1b1c4b34-407a-452b-9892-ccf9e68dd1e1\" (UID: \"1b1c4b34-407a-452b-9892-ccf9e68dd1e1\") "
Jul 24 14:43:37.814362 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:37.814339    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b1c4b34-407a-452b-9892-ccf9e68dd1e1-kube-api-access-8dpbj" (OuterVolumeSpecName: "kube-api-access-8dpbj") pod "1b1c4b34-407a-452b-9892-ccf9e68dd1e1" (UID: "1b1c4b34-407a-452b-9892-ccf9e68dd1e1"). InnerVolumeSpecName "kube-api-access-8dpbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:43:37.913342 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:37.913319    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dpbj\" (UniqueName: \"kubernetes.io/projected/1b1c4b34-407a-452b-9892-ccf9e68dd1e1-kube-api-access-8dpbj\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:43:38.483151 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:38.483122    2583 generic.go:356] "Generic (PLEG): container finished" podID="1b1c4b34-407a-452b-9892-ccf9e68dd1e1" containerID="29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5" exitCode=0
Jul 24 14:43:38.483533 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:38.483166    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hqmqc"
Jul 24 14:43:38.483533 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:38.483185    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hqmqc" event={"ID":"1b1c4b34-407a-452b-9892-ccf9e68dd1e1","Type":"ContainerDied","Data":"29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5"}
Jul 24 14:43:38.483533 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:38.483223    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hqmqc" event={"ID":"1b1c4b34-407a-452b-9892-ccf9e68dd1e1","Type":"ContainerDied","Data":"c2a427eac34b643db913ae5d21bf583363eabe542e92e1411d8863d762574cb8"}
Jul 24 14:43:38.483533 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:38.483246    2583 scope.go:117] "RemoveContainer" containerID="29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5"
Jul 24 14:43:38.491540 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:38.491519    2583 scope.go:117] "RemoveContainer" containerID="29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5"
Jul 24 14:43:38.491845 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:43:38.491819    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5\": container with ID starting with 29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5 not found: ID does not exist" containerID="29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5"
Jul 24 14:43:38.491899 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:38.491856    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5"} err="failed to get container status \"29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5\": rpc error: code = NotFound desc = could not find container \"29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5\": container with ID starting with 29e79b80c2e5b7dd15bd9199a05c50185ae3d40fcbe2fffa471b62c8526349f5 not found: ID does not exist"
Jul 24 14:43:38.505432 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:38.505408    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hqmqc"]
Jul 24 14:43:38.509151 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:38.509130    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hqmqc"]
Jul 24 14:43:39.345801 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:39.345775    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b1c4b34-407a-452b-9892-ccf9e68dd1e1" path="/var/lib/kubelet/pods/1b1c4b34-407a-452b-9892-ccf9e68dd1e1/volumes"
Jul 24 14:43:44.467479 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:44.467431    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:44.538356 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:44.538330    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"]
Jul 24 14:43:44.538529 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:44.538485    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq" podUID="8cc5444d-e793-427c-af6b-0ce781421418" containerName="limitador" containerID="cri-o://4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b" gracePeriod=30
Jul 24 14:43:45.073799 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.073777    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:45.159736 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.159686    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8cc5444d-e793-427c-af6b-0ce781421418-config-file\") pod \"8cc5444d-e793-427c-af6b-0ce781421418\" (UID: \"8cc5444d-e793-427c-af6b-0ce781421418\") "
Jul 24 14:43:45.159863 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.159741    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zckh8\" (UniqueName: \"kubernetes.io/projected/8cc5444d-e793-427c-af6b-0ce781421418-kube-api-access-zckh8\") pod \"8cc5444d-e793-427c-af6b-0ce781421418\" (UID: \"8cc5444d-e793-427c-af6b-0ce781421418\") "
Jul 24 14:43:45.160013 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.159988    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cc5444d-e793-427c-af6b-0ce781421418-config-file" (OuterVolumeSpecName: "config-file") pod "8cc5444d-e793-427c-af6b-0ce781421418" (UID: "8cc5444d-e793-427c-af6b-0ce781421418"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:43:45.161464 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.161433    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cc5444d-e793-427c-af6b-0ce781421418-kube-api-access-zckh8" (OuterVolumeSpecName: "kube-api-access-zckh8") pod "8cc5444d-e793-427c-af6b-0ce781421418" (UID: "8cc5444d-e793-427c-af6b-0ce781421418"). InnerVolumeSpecName "kube-api-access-zckh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:43:45.261205 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.261181    2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8cc5444d-e793-427c-af6b-0ce781421418-config-file\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:43:45.261298 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.261206    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zckh8\" (UniqueName: \"kubernetes.io/projected/8cc5444d-e793-427c-af6b-0ce781421418-kube-api-access-zckh8\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:43:45.511889 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.511834    2583 generic.go:356] "Generic (PLEG): container finished" podID="8cc5444d-e793-427c-af6b-0ce781421418" containerID="4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b" exitCode=0
Jul 24 14:43:45.511889 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.511872    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"
Jul 24 14:43:45.512219 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.511903    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq" event={"ID":"8cc5444d-e793-427c-af6b-0ce781421418","Type":"ContainerDied","Data":"4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b"}
Jul 24 14:43:45.512219 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.511941    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hfmrq" event={"ID":"8cc5444d-e793-427c-af6b-0ce781421418","Type":"ContainerDied","Data":"bcc7bbcbf0d6c58339629f758db488a221a9dce6d78f7b9faaa9e276a2bb2cc0"}
Jul 24 14:43:45.512219 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.511964    2583 scope.go:117] "RemoveContainer" containerID="4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b"
Jul 24 14:43:45.519546 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.519530    2583 scope.go:117] "RemoveContainer" containerID="4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b"
Jul 24 14:43:45.519798 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:43:45.519782    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b\": container with ID starting with 4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b not found: ID does not exist" containerID="4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b"
Jul 24 14:43:45.519847 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.519803    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b"} err="failed to get container status \"4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b\": rpc error: code = NotFound desc = could not find container \"4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b\": container with ID starting with 4da220b51fef1449cba0a64317421f8917a07d303cdb3cb728ca51848f21e98b not found: ID does not exist"
Jul 24 14:43:45.530823 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.530804    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"]
Jul 24 14:43:45.535216 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:45.535198    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hfmrq"]
Jul 24 14:43:47.341762 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:47.341733    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cc5444d-e793-427c-af6b-0ce781421418" path="/var/lib/kubelet/pods/8cc5444d-e793-427c-af6b-0ce781421418/volumes"
Jul 24 14:43:48.969722 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:48.969694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:49.069436 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.069414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:49.442491 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.442442    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"]
Jul 24 14:43:49.442785 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.442773    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc5444d-e793-427c-af6b-0ce781421418" containerName="limitador"
Jul 24 14:43:49.442821 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.442788    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc5444d-e793-427c-af6b-0ce781421418" containerName="limitador"
Jul 24 14:43:49.442821 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.442800    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b1c4b34-407a-452b-9892-ccf9e68dd1e1" containerName="authorino"
Jul 24 14:43:49.442821 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.442805    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b1c4b34-407a-452b-9892-ccf9e68dd1e1" containerName="authorino"
Jul 24 14:43:49.442918 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.442862    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b1c4b34-407a-452b-9892-ccf9e68dd1e1" containerName="authorino"
Jul 24 14:43:49.442918 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.442869    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc5444d-e793-427c-af6b-0ce781421418" containerName="limitador"
Jul 24 14:43:49.446848 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.446831    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:43:49.449110 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.449092    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 14:43:49.449928 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.449905    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-2mjp2\""
Jul 24 14:43:49.450014 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.449940    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:43:49.450105 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.450086    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 24 14:43:49.456080 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.456053    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"]
Jul 24 14:43:49.484720 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.484689    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:49.589347 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.589321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkfld\" (UniqueName: \"kubernetes.io/projected/fcf78e15-bd86-4e8d-82df-941790b7529a-kube-api-access-xkfld\") pod \"postgres-69fb4594cf-zbt9r\" (UID: \"fcf78e15-bd86-4e8d-82df-941790b7529a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:43:49.589435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.589352    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fcf78e15-bd86-4e8d-82df-941790b7529a-data\") pod \"postgres-69fb4594cf-zbt9r\" (UID: \"fcf78e15-bd86-4e8d-82df-941790b7529a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:43:49.690145 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.690124    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkfld\" (UniqueName: \"kubernetes.io/projected/fcf78e15-bd86-4e8d-82df-941790b7529a-kube-api-access-xkfld\") pod \"postgres-69fb4594cf-zbt9r\" (UID: \"fcf78e15-bd86-4e8d-82df-941790b7529a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:43:49.690248 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.690152    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fcf78e15-bd86-4e8d-82df-941790b7529a-data\") pod \"postgres-69fb4594cf-zbt9r\" (UID: \"fcf78e15-bd86-4e8d-82df-941790b7529a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:43:49.690441 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.690427    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fcf78e15-bd86-4e8d-82df-941790b7529a-data\") pod \"postgres-69fb4594cf-zbt9r\" (UID: \"fcf78e15-bd86-4e8d-82df-941790b7529a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:43:49.699252 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.699202    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkfld\" (UniqueName: \"kubernetes.io/projected/fcf78e15-bd86-4e8d-82df-941790b7529a-kube-api-access-xkfld\") pod \"postgres-69fb4594cf-zbt9r\" (UID: \"fcf78e15-bd86-4e8d-82df-941790b7529a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:43:49.762958 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.762940    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:43:49.886742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:49.886714    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"]
Jul 24 14:43:49.888647 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:43:49.888621    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf78e15_bd86_4e8d_82df_941790b7529a.slice/crio-bbe3399bed096cd518273fe371913ed3579968df69ae3d2059dd9e9cdbb2e7e8 WatchSource:0}: Error finding container bbe3399bed096cd518273fe371913ed3579968df69ae3d2059dd9e9cdbb2e7e8: Status 404 returned error can't find the container with id bbe3399bed096cd518273fe371913ed3579968df69ae3d2059dd9e9cdbb2e7e8
Jul 24 14:43:50.532239 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:50.532212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r" event={"ID":"fcf78e15-bd86-4e8d-82df-941790b7529a","Type":"ContainerStarted","Data":"bbe3399bed096cd518273fe371913ed3579968df69ae3d2059dd9e9cdbb2e7e8"}
Jul 24 14:43:51.636261 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:51.636225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:56.562622 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:56.562542    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r" event={"ID":"fcf78e15-bd86-4e8d-82df-941790b7529a","Type":"ContainerStarted","Data":"2a6db42a1df5ffb4b6dd16c61f81e04042e104148fb7dfa597d1b3b8937c33ab"}
Jul 24 14:43:56.562992 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:56.562638    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:43:56.581614 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:43:56.581572    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r" podStartSLOduration=1.353524291 podStartE2EDuration="7.581557467s" totalImagesPullingTime="6.228033176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:43:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:43:56.580185729 +0000 UTC m=+621.733186455" watchObservedRunningTime="2026-07-24 14:43:56.581557467 +0000 UTC m=+621.734558191"
Jul 24 14:44:02.591836 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:02.591771    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbt9r"
Jul 24 14:44:02.933264 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:02.933227    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:03.161909 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.161875    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-969dv"]
Jul 24 14:44:03.165399 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.165380    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-969dv"
Jul 24 14:44:03.168282 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.168266    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hpsrh\""
Jul 24 14:44:03.176577 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.176557    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-969dv"]
Jul 24 14:44:03.217823 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.217764    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:03.291577 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.291556    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4ph2\" (UniqueName: \"kubernetes.io/projected/ae6e4a68-4204-419a-8650-c4bc62f61f50-kube-api-access-f4ph2\") pod \"authorino-5d4b67c446-969dv\" (UID: \"ae6e4a68-4204-419a-8650-c4bc62f61f50\") " pod="kuadrant-system/authorino-5d4b67c446-969dv"
Jul 24 14:44:03.392360 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.392337    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4ph2\" (UniqueName: \"kubernetes.io/projected/ae6e4a68-4204-419a-8650-c4bc62f61f50-kube-api-access-f4ph2\") pod \"authorino-5d4b67c446-969dv\" (UID: \"ae6e4a68-4204-419a-8650-c4bc62f61f50\") " pod="kuadrant-system/authorino-5d4b67c446-969dv"
Jul 24 14:44:03.402683 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.402660    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4ph2\" (UniqueName: \"kubernetes.io/projected/ae6e4a68-4204-419a-8650-c4bc62f61f50-kube-api-access-f4ph2\") pod \"authorino-5d4b67c446-969dv\" (UID: \"ae6e4a68-4204-419a-8650-c4bc62f61f50\") " pod="kuadrant-system/authorino-5d4b67c446-969dv"
Jul 24 14:44:03.406867 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.406840    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-969dv"]
Jul 24 14:44:03.407046 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.407029    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-969dv"
Jul 24 14:44:03.429731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.429711    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6ff6867f74-mj8km"]
Jul 24 14:44:03.433003 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.432987    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ff6867f74-mj8km"
Jul 24 14:44:03.442304 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.442283    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6ff6867f74-mj8km"]
Jul 24 14:44:03.462689 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.462662    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6ff6867f74-mj8km"]
Jul 24 14:44:03.462955 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:44:03.462933    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7llxn], unattached volumes=[], failed to process volumes=[kube-api-access-7llxn]: context canceled" pod="kuadrant-system/authorino-6ff6867f74-mj8km" podUID="41e4a469-452c-4dd5-9f30-2064792ad211"
Jul 24 14:44:03.498018 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.497995    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c6fcb474-rkkfk"]
Jul 24 14:44:03.501234 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.501217    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:44:03.504741 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.504697    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 14:44:03.509444 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.509418    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c6fcb474-rkkfk"]
Jul 24 14:44:03.532950 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.532929    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-969dv"]
Jul 24 14:44:03.536721 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:44:03.536699    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae6e4a68_4204_419a_8650_c4bc62f61f50.slice/crio-c070d1af1f0af6e60875d19ce7aab031933cfd63a7961a823d1d1bbc803d70a3 WatchSource:0}: Error finding container c070d1af1f0af6e60875d19ce7aab031933cfd63a7961a823d1d1bbc803d70a3: Status 404 returned error can't find the container with id c070d1af1f0af6e60875d19ce7aab031933cfd63a7961a823d1d1bbc803d70a3
Jul 24 14:44:03.593507 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.593481    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7llxn\" (UniqueName: \"kubernetes.io/projected/41e4a469-452c-4dd5-9f30-2064792ad211-kube-api-access-7llxn\") pod \"authorino-6ff6867f74-mj8km\" (UID: \"41e4a469-452c-4dd5-9f30-2064792ad211\") " pod="kuadrant-system/authorino-6ff6867f74-mj8km"
Jul 24 14:44:03.593804 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.593510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ljl9\" (UniqueName: \"kubernetes.io/projected/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-kube-api-access-4ljl9\") pod \"authorino-c6fcb474-rkkfk\" (UID: \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\") " pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:44:03.593804 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.593558    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-tls-cert\") pod \"authorino-c6fcb474-rkkfk\" (UID: \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\") " pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:44:03.595986 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.595972    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ff6867f74-mj8km"
Jul 24 14:44:03.595986 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.595978    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-969dv" event={"ID":"ae6e4a68-4204-419a-8650-c4bc62f61f50","Type":"ContainerStarted","Data":"c070d1af1f0af6e60875d19ce7aab031933cfd63a7961a823d1d1bbc803d70a3"}
Jul 24 14:44:03.604006 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.603988    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ff6867f74-mj8km"
Jul 24 14:44:03.693893 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.693865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-tls-cert\") pod \"authorino-c6fcb474-rkkfk\" (UID: \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\") " pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:44:03.694065 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.693915    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7llxn\" (UniqueName: \"kubernetes.io/projected/41e4a469-452c-4dd5-9f30-2064792ad211-kube-api-access-7llxn\") pod \"authorino-6ff6867f74-mj8km\" (UID: \"41e4a469-452c-4dd5-9f30-2064792ad211\") " pod="kuadrant-system/authorino-6ff6867f74-mj8km"
Jul 24 14:44:03.694065 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.693933    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ljl9\" (UniqueName: \"kubernetes.io/projected/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-kube-api-access-4ljl9\") pod \"authorino-c6fcb474-rkkfk\" (UID: \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\") " pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:44:03.696060 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.696035    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-tls-cert\") pod \"authorino-c6fcb474-rkkfk\" (UID: \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\") " pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:44:03.702254 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.702230    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ljl9\" (UniqueName: \"kubernetes.io/projected/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-kube-api-access-4ljl9\") pod \"authorino-c6fcb474-rkkfk\" (UID: \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\") " pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:44:03.702337 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.702318    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7llxn\" (UniqueName: \"kubernetes.io/projected/41e4a469-452c-4dd5-9f30-2064792ad211-kube-api-access-7llxn\") pod \"authorino-6ff6867f74-mj8km\" (UID: \"41e4a469-452c-4dd5-9f30-2064792ad211\") " pod="kuadrant-system/authorino-6ff6867f74-mj8km"
Jul 24 14:44:03.794731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.794684    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7llxn\" (UniqueName: \"kubernetes.io/projected/41e4a469-452c-4dd5-9f30-2064792ad211-kube-api-access-7llxn\") pod \"41e4a469-452c-4dd5-9f30-2064792ad211\" (UID: \"41e4a469-452c-4dd5-9f30-2064792ad211\") "
Jul 24 14:44:03.796243 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.796223    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41e4a469-452c-4dd5-9f30-2064792ad211-kube-api-access-7llxn" (OuterVolumeSpecName: "kube-api-access-7llxn") pod "41e4a469-452c-4dd5-9f30-2064792ad211" (UID: "41e4a469-452c-4dd5-9f30-2064792ad211"). InnerVolumeSpecName "kube-api-access-7llxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:44:03.812378 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.812352    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:44:03.895974 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.895947    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7llxn\" (UniqueName: \"kubernetes.io/projected/41e4a469-452c-4dd5-9f30-2064792ad211-kube-api-access-7llxn\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:44:03.940710 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:03.938980    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c6fcb474-rkkfk"]
Jul 24 14:44:03.947670 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:44:03.947644    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb04dae4e_5430_4e9c_a9f5_8f95378fad3f.slice/crio-7f2ad49f5cca1023cee2c931c5818adccf23cdbcb3a9a48122e5d26a50b76be8 WatchSource:0}: Error finding container 7f2ad49f5cca1023cee2c931c5818adccf23cdbcb3a9a48122e5d26a50b76be8: Status 404 returned error can't find the container with id 7f2ad49f5cca1023cee2c931c5818adccf23cdbcb3a9a48122e5d26a50b76be8
Jul 24 14:44:04.601638 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:04.601611    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-969dv" event={"ID":"ae6e4a68-4204-419a-8650-c4bc62f61f50","Type":"ContainerStarted","Data":"8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d"}
Jul 24 14:44:04.601960 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:04.601652    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-969dv" podUID="ae6e4a68-4204-419a-8650-c4bc62f61f50" containerName="authorino" containerID="cri-o://8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d" gracePeriod=30
Jul 24 14:44:04.602989 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:04.602968    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c6fcb474-rkkfk" event={"ID":"b04dae4e-5430-4e9c-a9f5-8f95378fad3f","Type":"ContainerStarted","Data":"7f2ad49f5cca1023cee2c931c5818adccf23cdbcb3a9a48122e5d26a50b76be8"}
Jul 24 14:44:04.602989 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:04.602985    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ff6867f74-mj8km"
Jul 24 14:44:04.621865 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:04.621828    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-969dv" podStartSLOduration=1.209235075 podStartE2EDuration="1.62181461s" totalImagesPullingTime="412.579535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:44:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:44:04.619954898 +0000 UTC m=+629.772955622" watchObservedRunningTime="2026-07-24 14:44:04.62181461 +0000 UTC m=+629.774815394"
Jul 24 14:44:04.642815 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:04.642790    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6ff6867f74-mj8km"]
Jul 24 14:44:04.647524 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:04.647476    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6ff6867f74-mj8km"]
Jul 24 14:44:04.889536 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:04.889506    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-969dv"
Jul 24 14:44:05.003050 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.003021    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4ph2\" (UniqueName: \"kubernetes.io/projected/ae6e4a68-4204-419a-8650-c4bc62f61f50-kube-api-access-f4ph2\") pod \"ae6e4a68-4204-419a-8650-c4bc62f61f50\" (UID: \"ae6e4a68-4204-419a-8650-c4bc62f61f50\") "
Jul 24 14:44:05.004797 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.004776    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae6e4a68-4204-419a-8650-c4bc62f61f50-kube-api-access-f4ph2" (OuterVolumeSpecName: "kube-api-access-f4ph2") pod "ae6e4a68-4204-419a-8650-c4bc62f61f50" (UID: "ae6e4a68-4204-419a-8650-c4bc62f61f50"). InnerVolumeSpecName "kube-api-access-f4ph2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:44:05.104526 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.104507    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4ph2\" (UniqueName: \"kubernetes.io/projected/ae6e4a68-4204-419a-8650-c4bc62f61f50-kube-api-access-f4ph2\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:44:05.341789 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.341764    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41e4a469-452c-4dd5-9f30-2064792ad211" path="/var/lib/kubelet/pods/41e4a469-452c-4dd5-9f30-2064792ad211/volumes"
Jul 24 14:44:05.608217 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.608193    2583 generic.go:356] "Generic (PLEG): container finished" podID="ae6e4a68-4204-419a-8650-c4bc62f61f50" containerID="8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d" exitCode=0
Jul 24 14:44:05.608648 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.608245    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-969dv"
Jul 24 14:44:05.608648 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.608255    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-969dv" event={"ID":"ae6e4a68-4204-419a-8650-c4bc62f61f50","Type":"ContainerDied","Data":"8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d"}
Jul 24 14:44:05.608648 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.608289    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-969dv" event={"ID":"ae6e4a68-4204-419a-8650-c4bc62f61f50","Type":"ContainerDied","Data":"c070d1af1f0af6e60875d19ce7aab031933cfd63a7961a823d1d1bbc803d70a3"}
Jul 24 14:44:05.608648 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.608307    2583 scope.go:117] "RemoveContainer" containerID="8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d"
Jul 24 14:44:05.610029 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.610003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c6fcb474-rkkfk" event={"ID":"b04dae4e-5430-4e9c-a9f5-8f95378fad3f","Type":"ContainerStarted","Data":"777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07"}
Jul 24 14:44:05.616123 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.616108    2583 scope.go:117] "RemoveContainer" containerID="8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d"
Jul 24 14:44:05.616389 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:44:05.616372    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d\": container with ID starting with 8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d not found: ID does not exist" containerID="8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d"
Jul 24 14:44:05.616438 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.616396    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d"} err="failed to get container status \"8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d\": rpc error: code = NotFound desc = could not find container \"8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d\": container with ID starting with 8774733538aaa02969797a129a79ef2259b44960df585cf0a6fc5eb98bdf700d not found: ID does not exist"
Jul 24 14:44:05.631204 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.631164    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c6fcb474-rkkfk" podStartSLOduration=2.068432629 podStartE2EDuration="2.63115225s" totalImagesPullingTime="562.719621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:44:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:44:05.629046889 +0000 UTC m=+630.782047617" watchObservedRunningTime="2026-07-24 14:44:05.63115225 +0000 UTC m=+630.784152974"
Jul 24 14:44:05.643988 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.643968    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-969dv"]
Jul 24 14:44:05.648509 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:05.648489    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-969dv"]
Jul 24 14:44:07.342048 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:07.342019    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae6e4a68-4204-419a-8650-c4bc62f61f50" path="/var/lib/kubelet/pods/ae6e4a68-4204-419a-8650-c4bc62f61f50/volumes"
Jul 24 14:44:10.495759 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.495730    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74db68555b-57rkr"]
Jul 24 14:44:10.496090 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.496033    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae6e4a68-4204-419a-8650-c4bc62f61f50" containerName="authorino"
Jul 24 14:44:10.496090 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.496044    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae6e4a68-4204-419a-8650-c4bc62f61f50" containerName="authorino"
Jul 24 14:44:10.496090 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.496090    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae6e4a68-4204-419a-8650-c4bc62f61f50" containerName="authorino"
Jul 24 14:44:10.508560 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.508531    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74db68555b-57rkr"]
Jul 24 14:44:10.508714 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.508539    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:10.511533 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.511510    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 14:44:10.511896 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.511874    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8jvtl\""
Jul 24 14:44:10.512005 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.511881    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 14:44:10.643803 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.643776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfhjp\" (UniqueName: \"kubernetes.io/projected/6e7ba349-cbbb-4092-adc8-a0668a39201a-kube-api-access-vfhjp\") pod \"maas-controller-74db68555b-57rkr\" (UID: \"6e7ba349-cbbb-4092-adc8-a0668a39201a\") " pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:10.643925 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.643815    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e7ba349-cbbb-4092-adc8-a0668a39201a-webhook-cert\") pod \"maas-controller-74db68555b-57rkr\" (UID: \"6e7ba349-cbbb-4092-adc8-a0668a39201a\") " pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:10.673074 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.673048    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7955497cfb-spt6w"]
Jul 24 14:44:10.675462 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.675427    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:10.684732 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.684710    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7955497cfb-spt6w"]
Jul 24 14:44:10.744514 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.744447    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfhjp\" (UniqueName: \"kubernetes.io/projected/6e7ba349-cbbb-4092-adc8-a0668a39201a-kube-api-access-vfhjp\") pod \"maas-controller-74db68555b-57rkr\" (UID: \"6e7ba349-cbbb-4092-adc8-a0668a39201a\") " pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:10.744615 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.744561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e7ba349-cbbb-4092-adc8-a0668a39201a-webhook-cert\") pod \"maas-controller-74db68555b-57rkr\" (UID: \"6e7ba349-cbbb-4092-adc8-a0668a39201a\") " pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:10.746845 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.746793    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e7ba349-cbbb-4092-adc8-a0668a39201a-webhook-cert\") pod \"maas-controller-74db68555b-57rkr\" (UID: \"6e7ba349-cbbb-4092-adc8-a0668a39201a\") " pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:10.752081 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.752060    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfhjp\" (UniqueName: \"kubernetes.io/projected/6e7ba349-cbbb-4092-adc8-a0668a39201a-kube-api-access-vfhjp\") pod \"maas-controller-74db68555b-57rkr\" (UID: \"6e7ba349-cbbb-4092-adc8-a0668a39201a\") " pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:10.818896 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.818867    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:10.845148 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.845125    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a0d648b-2f0c-4bc9-9db0-9e147075a243-webhook-cert\") pod \"maas-controller-7955497cfb-spt6w\" (UID: \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\") " pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:10.845232 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.845196    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjj7\" (UniqueName: \"kubernetes.io/projected/1a0d648b-2f0c-4bc9-9db0-9e147075a243-kube-api-access-vnjj7\") pod \"maas-controller-7955497cfb-spt6w\" (UID: \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\") " pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:10.939286 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.939259    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74db68555b-57rkr"]
Jul 24 14:44:10.942644 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:44:10.942614    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7ba349_cbbb_4092_adc8_a0668a39201a.slice/crio-79dc8f087c94020e236e224d797f7903531e8bb48bc4131265b277ea111303b4 WatchSource:0}: Error finding container 79dc8f087c94020e236e224d797f7903531e8bb48bc4131265b277ea111303b4: Status 404 returned error can't find the container with id 79dc8f087c94020e236e224d797f7903531e8bb48bc4131265b277ea111303b4
Jul 24 14:44:10.945598 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.945567    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjj7\" (UniqueName: \"kubernetes.io/projected/1a0d648b-2f0c-4bc9-9db0-9e147075a243-kube-api-access-vnjj7\") pod \"maas-controller-7955497cfb-spt6w\" (UID: \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\") " pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:10.945704 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.945691    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a0d648b-2f0c-4bc9-9db0-9e147075a243-webhook-cert\") pod \"maas-controller-7955497cfb-spt6w\" (UID: \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\") " pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:10.947883 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.947858    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a0d648b-2f0c-4bc9-9db0-9e147075a243-webhook-cert\") pod \"maas-controller-7955497cfb-spt6w\" (UID: \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\") " pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:10.954462 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.954431    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjj7\" (UniqueName: \"kubernetes.io/projected/1a0d648b-2f0c-4bc9-9db0-9e147075a243-kube-api-access-vnjj7\") pod \"maas-controller-7955497cfb-spt6w\" (UID: \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\") " pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:10.985359 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:10.985341    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:11.105344 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:11.105320    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7955497cfb-spt6w"]
Jul 24 14:44:11.107585 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:44:11.107561    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0d648b_2f0c_4bc9_9db0_9e147075a243.slice/crio-0f3de1fa06166ed0a2a02238d640f5c4e026d1a112e0292423b67f89bea35076 WatchSource:0}: Error finding container 0f3de1fa06166ed0a2a02238d640f5c4e026d1a112e0292423b67f89bea35076: Status 404 returned error can't find the container with id 0f3de1fa06166ed0a2a02238d640f5c4e026d1a112e0292423b67f89bea35076
Jul 24 14:44:11.637373 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:11.637342    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-spt6w" event={"ID":"1a0d648b-2f0c-4bc9-9db0-9e147075a243","Type":"ContainerStarted","Data":"0f3de1fa06166ed0a2a02238d640f5c4e026d1a112e0292423b67f89bea35076"}
Jul 24 14:44:11.638739 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:11.638707    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74db68555b-57rkr" event={"ID":"6e7ba349-cbbb-4092-adc8-a0668a39201a","Type":"ContainerStarted","Data":"79dc8f087c94020e236e224d797f7903531e8bb48bc4131265b277ea111303b4"}
Jul 24 14:44:11.668958 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:11.668887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:11.731007 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:11.730977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:13.499886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:13.499861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:13.599398 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:13.599372    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:13.649701 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:13.649677    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-spt6w" event={"ID":"1a0d648b-2f0c-4bc9-9db0-9e147075a243","Type":"ContainerStarted","Data":"1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81"}
Jul 24 14:44:13.649820 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:13.649798    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:13.651140 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:13.651121    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74db68555b-57rkr" event={"ID":"6e7ba349-cbbb-4092-adc8-a0668a39201a","Type":"ContainerStarted","Data":"dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707"}
Jul 24 14:44:13.651246 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:13.651224    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:13.670524 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:13.670481    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7955497cfb-spt6w" podStartSLOduration=1.457312636 podStartE2EDuration="3.67046943s" totalImagesPullingTime="2.213156794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:44:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:44:13.669417443 +0000 UTC m=+638.822418167" watchObservedRunningTime="2026-07-24 14:44:13.67046943 +0000 UTC m=+638.823470150"
Jul 24 14:44:13.687390 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:13.687349    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74db68555b-57rkr" podStartSLOduration=1.131071733 podStartE2EDuration="3.687336837s" totalImagesPullingTime="2.556265104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:44:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:44:13.686644581 +0000 UTC m=+638.839645302" watchObservedRunningTime="2026-07-24 14:44:13.687336837 +0000 UTC m=+638.840337622"
Jul 24 14:44:24.658062 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:24.658034    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:24.658515 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:24.658094    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:44:24.723280 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:24.723248    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74db68555b-57rkr"]
Jul 24 14:44:24.723505 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:24.723468    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74db68555b-57rkr" podUID="6e7ba349-cbbb-4092-adc8-a0668a39201a" containerName="manager" containerID="cri-o://dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707" gracePeriod=30
Jul 24 14:44:24.952223 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:24.952201    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:25.150605 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.150577    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e7ba349-cbbb-4092-adc8-a0668a39201a-webhook-cert\") pod \"6e7ba349-cbbb-4092-adc8-a0668a39201a\" (UID: \"6e7ba349-cbbb-4092-adc8-a0668a39201a\") "
Jul 24 14:44:25.150743 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.150677    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfhjp\" (UniqueName: \"kubernetes.io/projected/6e7ba349-cbbb-4092-adc8-a0668a39201a-kube-api-access-vfhjp\") pod \"6e7ba349-cbbb-4092-adc8-a0668a39201a\" (UID: \"6e7ba349-cbbb-4092-adc8-a0668a39201a\") "
Jul 24 14:44:25.152419 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.152395    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e7ba349-cbbb-4092-adc8-a0668a39201a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6e7ba349-cbbb-4092-adc8-a0668a39201a" (UID: "6e7ba349-cbbb-4092-adc8-a0668a39201a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:44:25.152570 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.152549    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e7ba349-cbbb-4092-adc8-a0668a39201a-kube-api-access-vfhjp" (OuterVolumeSpecName: "kube-api-access-vfhjp") pod "6e7ba349-cbbb-4092-adc8-a0668a39201a" (UID: "6e7ba349-cbbb-4092-adc8-a0668a39201a"). InnerVolumeSpecName "kube-api-access-vfhjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:44:25.251443 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.251395    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfhjp\" (UniqueName: \"kubernetes.io/projected/6e7ba349-cbbb-4092-adc8-a0668a39201a-kube-api-access-vfhjp\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:44:25.251443 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.251416    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e7ba349-cbbb-4092-adc8-a0668a39201a-webhook-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:44:25.703449 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.703426    2583 generic.go:356] "Generic (PLEG): container finished" podID="6e7ba349-cbbb-4092-adc8-a0668a39201a" containerID="dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707" exitCode=0
Jul 24 14:44:25.703731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.703481    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74db68555b-57rkr"
Jul 24 14:44:25.703731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.703513    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74db68555b-57rkr" event={"ID":"6e7ba349-cbbb-4092-adc8-a0668a39201a","Type":"ContainerDied","Data":"dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707"}
Jul 24 14:44:25.703731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.703561    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74db68555b-57rkr" event={"ID":"6e7ba349-cbbb-4092-adc8-a0668a39201a","Type":"ContainerDied","Data":"79dc8f087c94020e236e224d797f7903531e8bb48bc4131265b277ea111303b4"}
Jul 24 14:44:25.703731 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.703581    2583 scope.go:117] "RemoveContainer" containerID="dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707"
Jul 24 14:44:25.710945 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.710929    2583 scope.go:117] "RemoveContainer" containerID="dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707"
Jul 24 14:44:25.711207 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:44:25.711187    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707\": container with ID starting with dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707 not found: ID does not exist" containerID="dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707"
Jul 24 14:44:25.711259 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.711215    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707"} err="failed to get container status \"dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707\": rpc error: code = NotFound desc = could not find container \"dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707\": container with ID starting with dfde46d9cbd22bf725d635b89f373d08922cfd2ba22578f6787c6793fde11707 not found: ID does not exist"
Jul 24 14:44:25.721386 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.721366    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74db68555b-57rkr"]
Jul 24 14:44:25.726547 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.726527    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74db68555b-57rkr"]
Jul 24 14:44:25.885814 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:25.885795    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:27.342148 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:27.342120    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e7ba349-cbbb-4092-adc8-a0668a39201a" path="/var/lib/kubelet/pods/6e7ba349-cbbb-4092-adc8-a0668a39201a/volumes"
Jul 24 14:44:28.631740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:28.631708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:28.689523 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:28.689495    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:44.063652 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.063606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:44.154391 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.154349    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:44.693964 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.693928    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:44.775376 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.775350    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"]
Jul 24 14:44:44.775705 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.775692    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e7ba349-cbbb-4092-adc8-a0668a39201a" containerName="manager"
Jul 24 14:44:44.775752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.775706    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e7ba349-cbbb-4092-adc8-a0668a39201a" containerName="manager"
Jul 24 14:44:44.775785 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.775758    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e7ba349-cbbb-4092-adc8-a0668a39201a" containerName="manager"
Jul 24 14:44:44.780042 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.780019    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:44.782227 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.782206    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 24 14:44:44.782349 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.782246    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d7qm7\""
Jul 24 14:44:44.791049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.791025    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"]
Jul 24 14:44:44.880195 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.880166    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k642\" (UniqueName: \"kubernetes.io/projected/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-kube-api-access-9k642\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:44.880321 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.880213    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-maas-api-tls\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:44.880321 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.880232    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-openshift-service-ca\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:44.981498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.981433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k642\" (UniqueName: \"kubernetes.io/projected/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-kube-api-access-9k642\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:44.981599 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.981507    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-maas-api-tls\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:44.981599 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.981523    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-openshift-service-ca\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:44.981669 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:44:44.981637    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 24 14:44:44.981715 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:44:44.981703    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-maas-api-tls podName:d48f106a-ac2b-49a6-b4c9-a35de5ffd414 nodeName:}" failed. No retries permitted until 2026-07-24 14:44:45.481682052 +0000 UTC m=+670.634682763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-maas-api-tls") pod "maas-api-79fb484fd4-72fc9" (UID: "d48f106a-ac2b-49a6-b4c9-a35de5ffd414") : secret "maas-api-serving-cert" not found
Jul 24 14:44:44.982181 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.982164    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-openshift-service-ca\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:44.991664 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:44.991631    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k642\" (UniqueName: \"kubernetes.io/projected/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-kube-api-access-9k642\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:45.199836 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.199783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:45.288747 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.288655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:45.374735 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.374707    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"]
Jul 24 14:44:45.382518 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.382495    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:44:45.385127 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.385101    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-k2hqh\""
Jul 24 14:44:45.385252 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.385103    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 14:44:45.386508 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.386490    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"]
Jul 24 14:44:45.484552 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.484522    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-maas-api-tls\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:45.484716 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.484579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bc47a37-2e1c-4a32-9536-9944afe9c4b5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-p6w7n\" (UID: \"5bc47a37-2e1c-4a32-9536-9944afe9c4b5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:44:45.484716 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.484611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77sxt\" (UniqueName: \"kubernetes.io/projected/5bc47a37-2e1c-4a32-9536-9944afe9c4b5-kube-api-access-77sxt\") pod \"payload-pre-processing-687dbb6cf6-p6w7n\" (UID: \"5bc47a37-2e1c-4a32-9536-9944afe9c4b5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:44:45.486889 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.486863    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d48f106a-ac2b-49a6-b4c9-a35de5ffd414-maas-api-tls\") pod \"maas-api-79fb484fd4-72fc9\" (UID: \"d48f106a-ac2b-49a6-b4c9-a35de5ffd414\") " pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:45.585803 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.585735    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bc47a37-2e1c-4a32-9536-9944afe9c4b5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-p6w7n\" (UID: \"5bc47a37-2e1c-4a32-9536-9944afe9c4b5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:44:45.585803 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.585775    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77sxt\" (UniqueName: \"kubernetes.io/projected/5bc47a37-2e1c-4a32-9536-9944afe9c4b5-kube-api-access-77sxt\") pod \"payload-pre-processing-687dbb6cf6-p6w7n\" (UID: \"5bc47a37-2e1c-4a32-9536-9944afe9c4b5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:44:45.586372 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.586355    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bc47a37-2e1c-4a32-9536-9944afe9c4b5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-p6w7n\" (UID: \"5bc47a37-2e1c-4a32-9536-9944afe9c4b5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:44:45.598258 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.598232    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77sxt\" (UniqueName: \"kubernetes.io/projected/5bc47a37-2e1c-4a32-9536-9944afe9c4b5-kube-api-access-77sxt\") pod \"payload-pre-processing-687dbb6cf6-p6w7n\" (UID: \"5bc47a37-2e1c-4a32-9536-9944afe9c4b5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:44:45.689487 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.689447    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:45.693715 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.693696    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:44:45.922604 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:45.922578    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"]
Jul 24 14:44:45.924599 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:44:45.924569    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc47a37_2e1c_4a32_9536_9944afe9c4b5.slice/crio-22a0b36b00681e98ad0a4a8f379fd1f09c329fb7a58ccd70d53af4f9bc698429 WatchSource:0}: Error finding container 22a0b36b00681e98ad0a4a8f379fd1f09c329fb7a58ccd70d53af4f9bc698429: Status 404 returned error can't find the container with id 22a0b36b00681e98ad0a4a8f379fd1f09c329fb7a58ccd70d53af4f9bc698429
Jul 24 14:44:46.085956 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:46.085932    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"]
Jul 24 14:44:46.794645 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:46.794604    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9" event={"ID":"d48f106a-ac2b-49a6-b4c9-a35de5ffd414","Type":"ContainerStarted","Data":"fb68d26d7dd8b0b7a784bb198988f8526e7eca98fd0fc3da720134e0d38f81d9"}
Jul 24 14:44:46.796029 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:46.796003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n" event={"ID":"5bc47a37-2e1c-4a32-9536-9944afe9c4b5","Type":"ContainerStarted","Data":"22a0b36b00681e98ad0a4a8f379fd1f09c329fb7a58ccd70d53af4f9bc698429"}
Jul 24 14:44:48.808752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:48.808715    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9" event={"ID":"d48f106a-ac2b-49a6-b4c9-a35de5ffd414","Type":"ContainerStarted","Data":"8034e1ae0fbad12e11d4b72883c5d117c2a4f41d85cccd245f9fd3e33d0f10ed"}
Jul 24 14:44:48.809182 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:48.808869    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:48.826700 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:48.826652    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9" podStartSLOduration=3.166142752 podStartE2EDuration="4.826633969s" totalImagesPullingTime="1.660491217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:44:48.825745509 +0000 UTC m=+673.978746233" watchObservedRunningTime="2026-07-24 14:44:48.826633969 +0000 UTC m=+673.979634693"
Jul 24 14:44:49.814427 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:49.814396    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n" event={"ID":"5bc47a37-2e1c-4a32-9536-9944afe9c4b5","Type":"ContainerStarted","Data":"64ca0abb466d319d16331eddd003c3316b72eb981ede69541bc3014975fd48b7"}
Jul 24 14:44:49.832152 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:49.832075    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n" podStartSLOduration=1.636953615 podStartE2EDuration="4.83205746s" totalImagesPullingTime="3.195103845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:44:49.831417096 +0000 UTC m=+674.984417821" watchObservedRunningTime="2026-07-24 14:44:49.83205746 +0000 UTC m=+674.985058188"
Jul 24 14:44:50.819710 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:50.819669    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:44:52.829484 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:52.829440    2583 generic.go:356] "Generic (PLEG): container finished" podID="040f935e-679e-4600-808b-aa9900fd2944" containerID="19d810872f4186830707e07eaf6af72da292c4e2ed3b719f99b1d62b44d241b8" exitCode=0
Jul 24 14:44:52.829811 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:52.829510    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" event={"ID":"040f935e-679e-4600-808b-aa9900fd2944","Type":"ContainerDied","Data":"19d810872f4186830707e07eaf6af72da292c4e2ed3b719f99b1d62b44d241b8"}
Jul 24 14:44:52.829858 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:52.829811    2583 scope.go:117] "RemoveContainer" containerID="19d810872f4186830707e07eaf6af72da292c4e2ed3b719f99b1d62b44d241b8"
Jul 24 14:44:52.830740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:52.830716    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:44:52.830815 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:52.830775    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:44:52.830815 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:52.830805    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 14:44:53.836043 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:53.836013    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" event={"ID":"040f935e-679e-4600-808b-aa9900fd2944","Type":"ContainerStarted","Data":"7ad24452b3f28ef5c8fda9419ed6a0d40a01a11f7cf46e8e7f4a695144e860fa"}
Jul 24 14:44:54.067989 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:54.067962    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:44:54.068109 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:54.068077    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-v8jkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 14:44:54.068150 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:54.068118    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" podUID="040f935e-679e-4600-808b-aa9900fd2944" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 14:44:54.820281 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:54.820250    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79fb484fd4-72fc9"
Jul 24 14:44:55.068740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:55.068705    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-v8jkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 14:44:55.069093 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:55.068761    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" podUID="040f935e-679e-4600-808b-aa9900fd2944" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 14:44:55.850491 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:55.850439    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:55.941324 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:55.941296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:56.068940 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:56.068903    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-v8jkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 14:44:56.069341 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:56.068961    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" podUID="040f935e-679e-4600-808b-aa9900fd2944" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 14:44:57.068447 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:57.068418    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-v8jkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 14:44:57.068627 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:57.068523    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" podUID="040f935e-679e-4600-808b-aa9900fd2944" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 14:44:58.068509 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:58.068438    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-v8jkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 14:44:58.068886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:58.068536    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd" podUID="040f935e-679e-4600-808b-aa9900fd2944" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 14:44:59.072005 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:59.071972    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:44:59.072344 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:59.072222    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:44:59.072809 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:44:59.072790    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd"
Jul 24 14:45:01.826405 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:01.826379    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n"
Jul 24 14:45:03.943110 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:03.943074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:45:09.253186 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:09.253152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:45:10.305937 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.305903    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"]
Jul 24 14:45:10.309408 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.309392    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.312495 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.312474    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kg9tm\""
Jul 24 14:45:10.312626 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.312510    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 24 14:45:10.312626 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.312574    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 14:45:10.312626 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.312513    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 14:45:10.324513 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.324495    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"]
Jul 24 14:45:10.373337 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.373312    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.373445 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.373340    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.373445 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.373355    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.373445 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.373420    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b57sf\" (UniqueName: \"kubernetes.io/projected/7e4108b9-bdcc-43b8-856d-89e57fa68901-kube-api-access-b57sf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.373641 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.373551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4108b9-bdcc-43b8-856d-89e57fa68901-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.373699 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.373641    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.414035 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.414007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:45:10.474448 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.474424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4108b9-bdcc-43b8-856d-89e57fa68901-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.474579 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.474487    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.474579 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.474517    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.474740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.474600    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.474740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.474638    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.474836 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.474736    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b57sf\" (UniqueName: \"kubernetes.io/projected/7e4108b9-bdcc-43b8-856d-89e57fa68901-kube-api-access-b57sf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.474907 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.474886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.474981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.474907    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.474981 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.474952    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.476784 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.476764    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4108b9-bdcc-43b8-856d-89e57fa68901-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.476913 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.476897    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4108b9-bdcc-43b8-856d-89e57fa68901-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.482998 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.482979    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b57sf\" (UniqueName: \"kubernetes.io/projected/7e4108b9-bdcc-43b8-856d-89e57fa68901-kube-api-access-b57sf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq\" (UID: \"7e4108b9-bdcc-43b8-856d-89e57fa68901\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.620106 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.620082    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:10.791946 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.791924    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"]
Jul 24 14:45:10.794369 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:45:10.794335    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4108b9_bdcc_43b8_856d_89e57fa68901.slice/crio-1eb5a0733eb6592e7ec0b3204ff6e0330b4b5a8346ef538bd1fb54f9b756931b WatchSource:0}: Error finding container 1eb5a0733eb6592e7ec0b3204ff6e0330b4b5a8346ef538bd1fb54f9b756931b: Status 404 returned error can't find the container with id 1eb5a0733eb6592e7ec0b3204ff6e0330b4b5a8346ef538bd1fb54f9b756931b
Jul 24 14:45:10.795888 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.795866    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:45:10.914141 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:10.914060    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq" event={"ID":"7e4108b9-bdcc-43b8-856d-89e57fa68901","Type":"ContainerStarted","Data":"1eb5a0733eb6592e7ec0b3204ff6e0330b4b5a8346ef538bd1fb54f9b756931b"}
Jul 24 14:45:12.926677 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:12.926601    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq" event={"ID":"7e4108b9-bdcc-43b8-856d-89e57fa68901","Type":"ContainerStarted","Data":"a50cccb37c5d5f874dc2733960a1e66852df75ef3318b07fdcd8739c4057a510"}
Jul 24 14:45:12.945422 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:12.945380    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq" podStartSLOduration=1.870983999 podStartE2EDuration="2.945360713s" totalImagesPullingTime="1.074376714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:45:12.944777623 +0000 UTC m=+698.097778349" watchObservedRunningTime="2026-07-24 14:45:12.945360713 +0000 UTC m=+698.098361439"
Jul 24 14:45:17.027764 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:17.027728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:45:19.684038 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.684005    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"]
Jul 24 14:45:19.688523 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.688499    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.690798 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.690776    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 24 14:45:19.697948 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.697921    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"]
Jul 24 14:45:19.719703 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.719676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:45:19.741638 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.741613    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.741740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.741657    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15db754d-858b-4643-9bb1-0655a0bdb571-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.741740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.741696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.741834 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.741759    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b7pb\" (UniqueName: \"kubernetes.io/projected/15db754d-858b-4643-9bb1-0655a0bdb571-kube-api-access-4b7pb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.741834 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.741798    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.741901 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.741850    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.842982 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.842950    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.843108 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.842991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15db754d-858b-4643-9bb1-0655a0bdb571-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.843108 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.843029    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.843108 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.843061    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b7pb\" (UniqueName: \"kubernetes.io/projected/15db754d-858b-4643-9bb1-0655a0bdb571-kube-api-access-4b7pb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.843108 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.843091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.843312 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.843133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.843508 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.843445    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.843508 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.843506    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.843738 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.843563    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.845209 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.845185    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15db754d-858b-4643-9bb1-0655a0bdb571-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.845583 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.845563    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15db754d-858b-4643-9bb1-0655a0bdb571-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.853770 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.853748    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b7pb\" (UniqueName: \"kubernetes.io/projected/15db754d-858b-4643-9bb1-0655a0bdb571-kube-api-access-4b7pb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4jtw5\" (UID: \"15db754d-858b-4643-9bb1-0655a0bdb571\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:19.999916 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:19.999866    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:20.130151 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:20.130126    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"]
Jul 24 14:45:20.131788 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:45:20.131752    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15db754d_858b_4643_9bb1_0655a0bdb571.slice/crio-e96c274d6bff163f0794fd53493cf4af5d5112f14bd3190d8b994dff07ee95f5 WatchSource:0}: Error finding container e96c274d6bff163f0794fd53493cf4af5d5112f14bd3190d8b994dff07ee95f5: Status 404 returned error can't find the container with id e96c274d6bff163f0794fd53493cf4af5d5112f14bd3190d8b994dff07ee95f5
Jul 24 14:45:20.620774 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:20.620745    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:20.620902 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:20.620797    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:20.632725 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:20.632704    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:20.964949 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:20.964874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5" event={"ID":"15db754d-858b-4643-9bb1-0655a0bdb571","Type":"ContainerStarted","Data":"54853fc3cbc1a8ecce3e3cb4e89721312a1232ae29c78d12dec5a7b58f54d12e"}
Jul 24 14:45:20.964949 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:20.964921    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5" event={"ID":"15db754d-858b-4643-9bb1-0655a0bdb571","Type":"ContainerStarted","Data":"e96c274d6bff163f0794fd53493cf4af5d5112f14bd3190d8b994dff07ee95f5"}
Jul 24 14:45:20.976972 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:20.976946    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq"
Jul 24 14:45:20.991058 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:20.991000    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5" podStartSLOduration=1.8028472089999998 podStartE2EDuration="1.990982341s" totalImagesPullingTime="188.135132ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:45:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:45:20.990147169 +0000 UTC m=+706.143147894" watchObservedRunningTime="2026-07-24 14:45:20.990982341 +0000 UTC m=+706.143983066"
Jul 24 14:45:30.000133 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:30.000043    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:30.000133 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:30.000108    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:30.011552 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:30.011532    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:30.113909 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:30.113876    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:45:31.017766 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:31.017735    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4jtw5"
Jul 24 14:45:38.416412 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:38.416376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:45:44.108624 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:45:44.108585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:33.051136 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.051104    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5846844f9f-jpfh2"]
Jul 24 14:46:33.054599 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.054575    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5846844f9f-jpfh2"
Jul 24 14:46:33.065998 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.065967    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5846844f9f-jpfh2"]
Jul 24 14:46:33.067795 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.067766    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgwtf\" (UniqueName: \"kubernetes.io/projected/62b14cb5-1877-4259-9464-db6f58c756d8-kube-api-access-pgwtf\") pod \"authorino-5846844f9f-jpfh2\" (UID: \"62b14cb5-1877-4259-9464-db6f58c756d8\") " pod="kuadrant-system/authorino-5846844f9f-jpfh2"
Jul 24 14:46:33.067906 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.067848    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/62b14cb5-1877-4259-9464-db6f58c756d8-tls-cert\") pod \"authorino-5846844f9f-jpfh2\" (UID: \"62b14cb5-1877-4259-9464-db6f58c756d8\") " pod="kuadrant-system/authorino-5846844f9f-jpfh2"
Jul 24 14:46:33.168280 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.168255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgwtf\" (UniqueName: \"kubernetes.io/projected/62b14cb5-1877-4259-9464-db6f58c756d8-kube-api-access-pgwtf\") pod \"authorino-5846844f9f-jpfh2\" (UID: \"62b14cb5-1877-4259-9464-db6f58c756d8\") " pod="kuadrant-system/authorino-5846844f9f-jpfh2"
Jul 24 14:46:33.168372 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.168294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/62b14cb5-1877-4259-9464-db6f58c756d8-tls-cert\") pod \"authorino-5846844f9f-jpfh2\" (UID: \"62b14cb5-1877-4259-9464-db6f58c756d8\") " pod="kuadrant-system/authorino-5846844f9f-jpfh2"
Jul 24 14:46:33.170890 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.170865    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/62b14cb5-1877-4259-9464-db6f58c756d8-tls-cert\") pod \"authorino-5846844f9f-jpfh2\" (UID: \"62b14cb5-1877-4259-9464-db6f58c756d8\") " pod="kuadrant-system/authorino-5846844f9f-jpfh2"
Jul 24 14:46:33.176806 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.176781    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgwtf\" (UniqueName: \"kubernetes.io/projected/62b14cb5-1877-4259-9464-db6f58c756d8-kube-api-access-pgwtf\") pod \"authorino-5846844f9f-jpfh2\" (UID: \"62b14cb5-1877-4259-9464-db6f58c756d8\") " pod="kuadrant-system/authorino-5846844f9f-jpfh2"
Jul 24 14:46:33.352363 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.352333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:33.366275 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.366251    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5846844f9f-jpfh2"
Jul 24 14:46:33.471107 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.471076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:33.517765 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:33.517741    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5846844f9f-jpfh2"]
Jul 24 14:46:33.520079 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:46:33.520038    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62b14cb5_1877_4259_9464_db6f58c756d8.slice/crio-f96e0de8e427282a7e0d6613dde089efea7700fdd8127197d580e7a97397f595 WatchSource:0}: Error finding container f96e0de8e427282a7e0d6613dde089efea7700fdd8127197d580e7a97397f595: Status 404 returned error can't find the container with id f96e0de8e427282a7e0d6613dde089efea7700fdd8127197d580e7a97397f595
Jul 24 14:46:34.287940 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.287909    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5846844f9f-jpfh2" event={"ID":"62b14cb5-1877-4259-9464-db6f58c756d8","Type":"ContainerStarted","Data":"f93dd9c47d13e01153c7ce259d6e8a362f3420a790321e1b88d43c12ee2daa2c"}
Jul 24 14:46:34.288275 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.287947    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5846844f9f-jpfh2" event={"ID":"62b14cb5-1877-4259-9464-db6f58c756d8","Type":"ContainerStarted","Data":"f96e0de8e427282a7e0d6613dde089efea7700fdd8127197d580e7a97397f595"}
Jul 24 14:46:34.312384 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.312333    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5846844f9f-jpfh2" podStartSLOduration=0.646836329 podStartE2EDuration="1.312312465s" totalImagesPullingTime="665.476136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:46:34.31069352 +0000 UTC m=+779.463694247" watchObservedRunningTime="2026-07-24 14:46:34.312312465 +0000 UTC m=+779.465313191"
Jul 24 14:46:34.350251 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.350216    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c6fcb474-rkkfk"]
Jul 24 14:46:34.350440 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.350412    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c6fcb474-rkkfk" podUID="b04dae4e-5430-4e9c-a9f5-8f95378fad3f" containerName="authorino" containerID="cri-o://777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07" gracePeriod=30
Jul 24 14:46:34.598305 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.598278    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:46:34.681869 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.681837    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-tls-cert\") pod \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\" (UID: \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\") "
Jul 24 14:46:34.682014 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.681905    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ljl9\" (UniqueName: \"kubernetes.io/projected/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-kube-api-access-4ljl9\") pod \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\" (UID: \"b04dae4e-5430-4e9c-a9f5-8f95378fad3f\") "
Jul 24 14:46:34.684031 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.683999    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-kube-api-access-4ljl9" (OuterVolumeSpecName: "kube-api-access-4ljl9") pod "b04dae4e-5430-4e9c-a9f5-8f95378fad3f" (UID: "b04dae4e-5430-4e9c-a9f5-8f95378fad3f"). InnerVolumeSpecName "kube-api-access-4ljl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:46:34.695516 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.695471    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b04dae4e-5430-4e9c-a9f5-8f95378fad3f" (UID: "b04dae4e-5430-4e9c-a9f5-8f95378fad3f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:46:34.758513 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.758414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:34.782820 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.782795    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ljl9\" (UniqueName: \"kubernetes.io/projected/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-kube-api-access-4ljl9\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:46:34.782820 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.782817    2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b04dae4e-5430-4e9c-a9f5-8f95378fad3f-tls-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:46:34.937703 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:34.937666    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:35.294594 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.294557    2583 generic.go:356] "Generic (PLEG): container finished" podID="b04dae4e-5430-4e9c-a9f5-8f95378fad3f" containerID="777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07" exitCode=0
Jul 24 14:46:35.295218 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.295192    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c6fcb474-rkkfk"
Jul 24 14:46:35.295368 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.295340    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c6fcb474-rkkfk" event={"ID":"b04dae4e-5430-4e9c-a9f5-8f95378fad3f","Type":"ContainerDied","Data":"777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07"}
Jul 24 14:46:35.295501 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.295396    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c6fcb474-rkkfk" event={"ID":"b04dae4e-5430-4e9c-a9f5-8f95378fad3f","Type":"ContainerDied","Data":"7f2ad49f5cca1023cee2c931c5818adccf23cdbcb3a9a48122e5d26a50b76be8"}
Jul 24 14:46:35.295501 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.295418    2583 scope.go:117] "RemoveContainer" containerID="777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07"
Jul 24 14:46:35.303532 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.303515    2583 scope.go:117] "RemoveContainer" containerID="777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07"
Jul 24 14:46:35.303772 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:46:35.303755    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07\": container with ID starting with 777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07 not found: ID does not exist" containerID="777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07"
Jul 24 14:46:35.303827 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.303781    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07"} err="failed to get container status \"777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07\": rpc error: code = NotFound desc = could not find container \"777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07\": container with ID starting with 777699a4954f61fae1eb27b89b31e5a0372c0cb090a761d0d73a3c1a50ba1d07 not found: ID does not exist"
Jul 24 14:46:35.342691 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.342668    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c6fcb474-rkkfk"]
Jul 24 14:46:35.353545 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.353516    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c6fcb474-rkkfk"]
Jul 24 14:46:35.537904 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.537821    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:35.686808 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:35.686764    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:37.342245 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:46:37.342216    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b04dae4e-5430-4e9c-a9f5-8f95378fad3f" path="/var/lib/kubelet/pods/b04dae4e-5430-4e9c-a9f5-8f95378fad3f/volumes"
Jul 24 14:47:21.161895 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.161859    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7955497cfb-spt6w"]
Jul 24 14:47:21.162408 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.162080    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7955497cfb-spt6w" podUID="1a0d648b-2f0c-4bc9-9db0-9e147075a243" containerName="manager" containerID="cri-o://1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81" gracePeriod=30
Jul 24 14:47:21.403839 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.403816    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:47:21.504473 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.504382    2583 generic.go:356] "Generic (PLEG): container finished" podID="1a0d648b-2f0c-4bc9-9db0-9e147075a243" containerID="1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81" exitCode=0
Jul 24 14:47:21.504473 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.504434    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-spt6w"
Jul 24 14:47:21.504632 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.504487    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-spt6w" event={"ID":"1a0d648b-2f0c-4bc9-9db0-9e147075a243","Type":"ContainerDied","Data":"1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81"}
Jul 24 14:47:21.504632 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.504549    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-spt6w" event={"ID":"1a0d648b-2f0c-4bc9-9db0-9e147075a243","Type":"ContainerDied","Data":"0f3de1fa06166ed0a2a02238d640f5c4e026d1a112e0292423b67f89bea35076"}
Jul 24 14:47:21.504632 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.504580    2583 scope.go:117] "RemoveContainer" containerID="1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81"
Jul 24 14:47:21.513344 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.513317    2583 scope.go:117] "RemoveContainer" containerID="1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81"
Jul 24 14:47:21.513622 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:47:21.513602    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81\": container with ID starting with 1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81 not found: ID does not exist" containerID="1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81"
Jul 24 14:47:21.513728 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.513628    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81"} err="failed to get container status \"1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81\": rpc error: code = NotFound desc = could not find container \"1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81\": container with ID starting with 1debcd98a620d6e2f2b41bf3dda38073f037cec07054c51862cccbec9ab03a81 not found: ID does not exist"
Jul 24 14:47:21.532022 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.532000    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnjj7\" (UniqueName: \"kubernetes.io/projected/1a0d648b-2f0c-4bc9-9db0-9e147075a243-kube-api-access-vnjj7\") pod \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\" (UID: \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\") "
Jul 24 14:47:21.532118 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.532072    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a0d648b-2f0c-4bc9-9db0-9e147075a243-webhook-cert\") pod \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\" (UID: \"1a0d648b-2f0c-4bc9-9db0-9e147075a243\") "
Jul 24 14:47:21.533918 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.533893    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a0d648b-2f0c-4bc9-9db0-9e147075a243-kube-api-access-vnjj7" (OuterVolumeSpecName: "kube-api-access-vnjj7") pod "1a0d648b-2f0c-4bc9-9db0-9e147075a243" (UID: "1a0d648b-2f0c-4bc9-9db0-9e147075a243"). InnerVolumeSpecName "kube-api-access-vnjj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:47:21.534006 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.533965    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a0d648b-2f0c-4bc9-9db0-9e147075a243-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1a0d648b-2f0c-4bc9-9db0-9e147075a243" (UID: "1a0d648b-2f0c-4bc9-9db0-9e147075a243"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:47:21.633460 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.633437    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnjj7\" (UniqueName: \"kubernetes.io/projected/1a0d648b-2f0c-4bc9-9db0-9e147075a243-kube-api-access-vnjj7\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:47:21.633548 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.633470    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a0d648b-2f0c-4bc9-9db0-9e147075a243-webhook-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:47:21.829125 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.829102    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7955497cfb-spt6w"]
Jul 24 14:47:21.833660 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:21.833639    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7955497cfb-spt6w"]
Jul 24 14:47:23.344500 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:23.344449    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a0d648b-2f0c-4bc9-9db0-9e147075a243" path="/var/lib/kubelet/pods/1a0d648b-2f0c-4bc9-9db0-9e147075a243/volumes"
Jul 24 14:47:29.252976 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:47:29.252940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:48:17.401742 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:17.401708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:48:17.471237 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:17.471201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:48:22.708082 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.708053    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7955497cfb-z6kpj"]
Jul 24 14:48:22.708498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.708365    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b04dae4e-5430-4e9c-a9f5-8f95378fad3f" containerName="authorino"
Jul 24 14:48:22.708498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.708375    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b04dae4e-5430-4e9c-a9f5-8f95378fad3f" containerName="authorino"
Jul 24 14:48:22.708498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.708394    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0d648b-2f0c-4bc9-9db0-9e147075a243" containerName="manager"
Jul 24 14:48:22.708498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.708399    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0d648b-2f0c-4bc9-9db0-9e147075a243" containerName="manager"
Jul 24 14:48:22.708498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.708476    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b04dae4e-5430-4e9c-a9f5-8f95378fad3f" containerName="authorino"
Jul 24 14:48:22.708498 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.708488    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a0d648b-2f0c-4bc9-9db0-9e147075a243" containerName="manager"
Jul 24 14:48:22.711474 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.711441    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:22.715733 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.715710    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8jvtl\""
Jul 24 14:48:22.715733 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.715736    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 14:48:22.715969 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.715956    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 14:48:22.721845 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.721822    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7955497cfb-z6kpj"]
Jul 24 14:48:22.767218 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.767195    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kshzk\" (UniqueName: \"kubernetes.io/projected/9275a9dc-edcf-4294-92d9-3041f8989cd8-kube-api-access-kshzk\") pod \"maas-controller-7955497cfb-z6kpj\" (UID: \"9275a9dc-edcf-4294-92d9-3041f8989cd8\") " pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:22.767319 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.767274    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9275a9dc-edcf-4294-92d9-3041f8989cd8-webhook-cert\") pod \"maas-controller-7955497cfb-z6kpj\" (UID: \"9275a9dc-edcf-4294-92d9-3041f8989cd8\") " pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:22.868120 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.868093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9275a9dc-edcf-4294-92d9-3041f8989cd8-webhook-cert\") pod \"maas-controller-7955497cfb-z6kpj\" (UID: \"9275a9dc-edcf-4294-92d9-3041f8989cd8\") " pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:22.868207 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.868131    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kshzk\" (UniqueName: \"kubernetes.io/projected/9275a9dc-edcf-4294-92d9-3041f8989cd8-kube-api-access-kshzk\") pod \"maas-controller-7955497cfb-z6kpj\" (UID: \"9275a9dc-edcf-4294-92d9-3041f8989cd8\") " pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:22.870320 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.870300    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9275a9dc-edcf-4294-92d9-3041f8989cd8-webhook-cert\") pod \"maas-controller-7955497cfb-z6kpj\" (UID: \"9275a9dc-edcf-4294-92d9-3041f8989cd8\") " pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:22.877106 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:22.877088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kshzk\" (UniqueName: \"kubernetes.io/projected/9275a9dc-edcf-4294-92d9-3041f8989cd8-kube-api-access-kshzk\") pod \"maas-controller-7955497cfb-z6kpj\" (UID: \"9275a9dc-edcf-4294-92d9-3041f8989cd8\") " pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:23.048911 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:23.048860    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:23.173781 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:23.173757    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7955497cfb-z6kpj"]
Jul 24 14:48:23.176285 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:48:23.176263    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9275a9dc_edcf_4294_92d9_3041f8989cd8.slice/crio-1f69d8886c2b0e9de514c880fd44138b1a1b8be8909a5b9c07faa526ef6e34de WatchSource:0}: Error finding container 1f69d8886c2b0e9de514c880fd44138b1a1b8be8909a5b9c07faa526ef6e34de: Status 404 returned error can't find the container with id 1f69d8886c2b0e9de514c880fd44138b1a1b8be8909a5b9c07faa526ef6e34de
Jul 24 14:48:23.774474 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:23.774427    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-z6kpj" event={"ID":"9275a9dc-edcf-4294-92d9-3041f8989cd8","Type":"ContainerStarted","Data":"1f69d8886c2b0e9de514c880fd44138b1a1b8be8909a5b9c07faa526ef6e34de"}
Jul 24 14:48:24.780826 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:24.780791    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-z6kpj" event={"ID":"9275a9dc-edcf-4294-92d9-3041f8989cd8","Type":"ContainerStarted","Data":"f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5"}
Jul 24 14:48:24.781193 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:24.780883    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:24.800559 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:24.800514    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7955497cfb-z6kpj" podStartSLOduration=2.027039946 podStartE2EDuration="2.800498333s" totalImagesPullingTime="773.458387ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:48:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:48:24.79952905 +0000 UTC m=+889.952529774" watchObservedRunningTime="2026-07-24 14:48:24.800498333 +0000 UTC m=+889.953499054"
Jul 24 14:48:35.788148 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:35.788079    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:40.351049 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.351012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:48:40.448109 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.448078    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7955497cfb-z6kpj"]
Jul 24 14:48:40.448293 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.448253    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7955497cfb-z6kpj" podUID="9275a9dc-edcf-4294-92d9-3041f8989cd8" containerName="manager" containerID="cri-o://f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5" gracePeriod=30
Jul 24 14:48:40.679967 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.679942    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:40.807047 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.807022    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kshzk\" (UniqueName: \"kubernetes.io/projected/9275a9dc-edcf-4294-92d9-3041f8989cd8-kube-api-access-kshzk\") pod \"9275a9dc-edcf-4294-92d9-3041f8989cd8\" (UID: \"9275a9dc-edcf-4294-92d9-3041f8989cd8\") "
Jul 24 14:48:40.807197 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.807087    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9275a9dc-edcf-4294-92d9-3041f8989cd8-webhook-cert\") pod \"9275a9dc-edcf-4294-92d9-3041f8989cd8\" (UID: \"9275a9dc-edcf-4294-92d9-3041f8989cd8\") "
Jul 24 14:48:40.808870 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.808838    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9275a9dc-edcf-4294-92d9-3041f8989cd8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9275a9dc-edcf-4294-92d9-3041f8989cd8" (UID: "9275a9dc-edcf-4294-92d9-3041f8989cd8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:48:40.808965 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.808886    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9275a9dc-edcf-4294-92d9-3041f8989cd8-kube-api-access-kshzk" (OuterVolumeSpecName: "kube-api-access-kshzk") pod "9275a9dc-edcf-4294-92d9-3041f8989cd8" (UID: "9275a9dc-edcf-4294-92d9-3041f8989cd8"). InnerVolumeSpecName "kube-api-access-kshzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:48:40.853491 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.853464    2583 generic.go:356] "Generic (PLEG): container finished" podID="9275a9dc-edcf-4294-92d9-3041f8989cd8" containerID="f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5" exitCode=0
Jul 24 14:48:40.853601 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.853513    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-z6kpj"
Jul 24 14:48:40.853601 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.853522    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-z6kpj" event={"ID":"9275a9dc-edcf-4294-92d9-3041f8989cd8","Type":"ContainerDied","Data":"f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5"}
Jul 24 14:48:40.853601 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.853582    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-z6kpj" event={"ID":"9275a9dc-edcf-4294-92d9-3041f8989cd8","Type":"ContainerDied","Data":"1f69d8886c2b0e9de514c880fd44138b1a1b8be8909a5b9c07faa526ef6e34de"}
Jul 24 14:48:40.853762 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.853616    2583 scope.go:117] "RemoveContainer" containerID="f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5"
Jul 24 14:48:40.864349 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.864333    2583 scope.go:117] "RemoveContainer" containerID="f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5"
Jul 24 14:48:40.864610 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:48:40.864594    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5\": container with ID starting with f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5 not found: ID does not exist" containerID="f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5"
Jul 24 14:48:40.864675 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.864626    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5"} err="failed to get container status \"f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5\": rpc error: code = NotFound desc = could not find container \"f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5\": container with ID starting with f4544e5484fb521c51dccbf6f895cf265653fd86d9f591fa73cb9b2dc72bd4e5 not found: ID does not exist"
Jul 24 14:48:40.879054 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.879030    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7955497cfb-z6kpj"]
Jul 24 14:48:40.883056 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.883037    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7955497cfb-z6kpj"]
Jul 24 14:48:40.907886 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.907868    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kshzk\" (UniqueName: \"kubernetes.io/projected/9275a9dc-edcf-4294-92d9-3041f8989cd8-kube-api-access-kshzk\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:48:40.907949 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:40.907886    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9275a9dc-edcf-4294-92d9-3041f8989cd8-webhook-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:48:41.344258 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.344236    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9275a9dc-edcf-4294-92d9-3041f8989cd8" path="/var/lib/kubelet/pods/9275a9dc-edcf-4294-92d9-3041f8989cd8/volumes"
Jul 24 14:48:41.619482 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.619403    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:48:41.735278 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.735250    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7955497cfb-b4g79"]
Jul 24 14:48:41.735667 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.735650    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9275a9dc-edcf-4294-92d9-3041f8989cd8" containerName="manager"
Jul 24 14:48:41.735754 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.735669    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9275a9dc-edcf-4294-92d9-3041f8989cd8" containerName="manager"
Jul 24 14:48:41.735812 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.735780    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9275a9dc-edcf-4294-92d9-3041f8989cd8" containerName="manager"
Jul 24 14:48:41.739975 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.739956    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:48:41.742968 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.742942    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 14:48:41.742968 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.742942    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8jvtl\""
Jul 24 14:48:41.743119 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.743000    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 14:48:41.748085 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.748063    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7955497cfb-b4g79"]
Jul 24 14:48:41.813736 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.813713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07e49534-6a68-4595-a3ac-ba93fbfd98e6-webhook-cert\") pod \"maas-controller-7955497cfb-b4g79\" (UID: \"07e49534-6a68-4595-a3ac-ba93fbfd98e6\") " pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:48:41.813824 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.813753    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbzvk\" (UniqueName: \"kubernetes.io/projected/07e49534-6a68-4595-a3ac-ba93fbfd98e6-kube-api-access-bbzvk\") pod \"maas-controller-7955497cfb-b4g79\" (UID: \"07e49534-6a68-4595-a3ac-ba93fbfd98e6\") " pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:48:41.914422 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.914360    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07e49534-6a68-4595-a3ac-ba93fbfd98e6-webhook-cert\") pod \"maas-controller-7955497cfb-b4g79\" (UID: \"07e49534-6a68-4595-a3ac-ba93fbfd98e6\") " pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:48:41.914422 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.914415    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbzvk\" (UniqueName: \"kubernetes.io/projected/07e49534-6a68-4595-a3ac-ba93fbfd98e6-kube-api-access-bbzvk\") pod \"maas-controller-7955497cfb-b4g79\" (UID: \"07e49534-6a68-4595-a3ac-ba93fbfd98e6\") " pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:48:41.916578 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.916558    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07e49534-6a68-4595-a3ac-ba93fbfd98e6-webhook-cert\") pod \"maas-controller-7955497cfb-b4g79\" (UID: \"07e49534-6a68-4595-a3ac-ba93fbfd98e6\") " pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:48:41.922812 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:41.922791    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbzvk\" (UniqueName: \"kubernetes.io/projected/07e49534-6a68-4595-a3ac-ba93fbfd98e6-kube-api-access-bbzvk\") pod \"maas-controller-7955497cfb-b4g79\" (UID: \"07e49534-6a68-4595-a3ac-ba93fbfd98e6\") " pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:48:42.051020 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:42.051002    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:48:42.175812 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:42.175783    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7955497cfb-b4g79"]
Jul 24 14:48:42.178178 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:48:42.178147    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e49534_6a68_4595_a3ac_ba93fbfd98e6.slice/crio-3e178e954b406f5dfbba3043e9174063ce00834aa2edb2aa7524545f94ee2c29 WatchSource:0}: Error finding container 3e178e954b406f5dfbba3043e9174063ce00834aa2edb2aa7524545f94ee2c29: Status 404 returned error can't find the container with id 3e178e954b406f5dfbba3043e9174063ce00834aa2edb2aa7524545f94ee2c29
Jul 24 14:48:42.864634 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:42.864596    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-b4g79" event={"ID":"07e49534-6a68-4595-a3ac-ba93fbfd98e6","Type":"ContainerStarted","Data":"d8e5f2034c1fb5cf2f1b6736bddb44ac6202f99c1fbd6125b0d331638300f1b8"}
Jul 24 14:48:42.864987 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:42.864657    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7955497cfb-b4g79" event={"ID":"07e49534-6a68-4595-a3ac-ba93fbfd98e6","Type":"ContainerStarted","Data":"3e178e954b406f5dfbba3043e9174063ce00834aa2edb2aa7524545f94ee2c29"}
Jul 24 14:48:42.864987 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:42.864689    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:48:42.888444 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:42.884177    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7955497cfb-b4g79" podStartSLOduration=1.476764106 podStartE2EDuration="1.884159604s" totalImagesPullingTime="407.395498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:48:42.881136708 +0000 UTC m=+908.034137444" watchObservedRunningTime="2026-07-24 14:48:42.884159604 +0000 UTC m=+908.037160330"
Jul 24 14:48:53.870813 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:48:53.870781    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7955497cfb-b4g79"
Jul 24 14:49:01.184200 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:01.184146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:01.255830 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:01.255802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:11.927646 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:11.927611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:17.334077 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:17.334050    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:17.421926 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:17.421897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:17.556882 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:17.556840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:23.744230 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:23.744197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:23.845941 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:23.845904    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:23.947488 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:23.947436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:24.092264 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:24.092196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:30.449589 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:30.449556    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:30.519433 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:30.519407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:45.089656 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:49:45.089597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:52:53.043403 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:52:53.043363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:52:54.567107 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:52:54.567077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:53:03.483882 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:03.483746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:53:03.585651 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:03.585618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:53:03.685844 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:03.685817    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:53:31.092310 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:31.092273    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"]
Jul 24 14:53:31.093174 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:31.093140    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l" podUID="511360df-9654-4ab9-a5ba-70bdb1512d21" containerName="manager" containerID="cri-o://38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649" gracePeriod=10
Jul 24 14:53:33.347993 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:33.347966    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:53:33.427740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:33.427683    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/511360df-9654-4ab9-a5ba-70bdb1512d21-extensions-socket-volume\") pod \"511360df-9654-4ab9-a5ba-70bdb1512d21\" (UID: \"511360df-9654-4ab9-a5ba-70bdb1512d21\") "
Jul 24 14:53:33.427740 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:33.427722    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44cjb\" (UniqueName: \"kubernetes.io/projected/511360df-9654-4ab9-a5ba-70bdb1512d21-kube-api-access-44cjb\") pod \"511360df-9654-4ab9-a5ba-70bdb1512d21\" (UID: \"511360df-9654-4ab9-a5ba-70bdb1512d21\") "
Jul 24 14:53:33.427998 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:33.427967    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511360df-9654-4ab9-a5ba-70bdb1512d21-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "511360df-9654-4ab9-a5ba-70bdb1512d21" (UID: "511360df-9654-4ab9-a5ba-70bdb1512d21"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:53:33.429478 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:33.429443    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/511360df-9654-4ab9-a5ba-70bdb1512d21-kube-api-access-44cjb" (OuterVolumeSpecName: "kube-api-access-44cjb") pod "511360df-9654-4ab9-a5ba-70bdb1512d21" (UID: "511360df-9654-4ab9-a5ba-70bdb1512d21"). InnerVolumeSpecName "kube-api-access-44cjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:53:33.529166 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:33.529146    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/511360df-9654-4ab9-a5ba-70bdb1512d21-extensions-socket-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:53:33.529166 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:33.529166    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44cjb\" (UniqueName: \"kubernetes.io/projected/511360df-9654-4ab9-a5ba-70bdb1512d21-kube-api-access-44cjb\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:53:34.134422 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:34.134397    2583 generic.go:356] "Generic (PLEG): container finished" podID="511360df-9654-4ab9-a5ba-70bdb1512d21" containerID="38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649" exitCode=0
Jul 24 14:53:34.134556 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:34.134449    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"
Jul 24 14:53:34.134556 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:34.134482    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l" event={"ID":"511360df-9654-4ab9-a5ba-70bdb1512d21","Type":"ContainerDied","Data":"38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649"}
Jul 24 14:53:34.134556 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:34.134534    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l" event={"ID":"511360df-9654-4ab9-a5ba-70bdb1512d21","Type":"ContainerDied","Data":"172f8e4a425de5200c0ed06f818bcf8ac015fd7d0d145e05881d8a3bdb4c2056"}
Jul 24 14:53:34.134556 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:34.134554    2583 scope.go:117] "RemoveContainer" containerID="38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649"
Jul 24 14:53:34.143690 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:34.143674    2583 scope.go:117] "RemoveContainer" containerID="38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649"
Jul 24 14:53:34.143938 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:53:34.143921    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649\": container with ID starting with 38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649 not found: ID does not exist" containerID="38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649"
Jul 24 14:53:34.143982 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:34.143947    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649"} err="failed to get container status \"38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649\": rpc error: code = NotFound desc = could not find container \"38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649\": container with ID starting with 38ab0547b56c2f84d8eb9dbc3beb2b587403ebaa33a4af98d8a8a49470924649 not found: ID does not exist"
Jul 24 14:53:34.155726 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:34.155709    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"]
Jul 24 14:53:34.160048 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:34.160028    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bwb4l"]
Jul 24 14:53:35.341854 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:53:35.341825    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="511360df-9654-4ab9-a5ba-70bdb1512d21" path="/var/lib/kubelet/pods/511360df-9654-4ab9-a5ba-70bdb1512d21/volumes"
Jul 24 14:54:08.272033 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:08.272004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:54:32.218662 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.218587    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"]
Jul 24 14:54:32.218982 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.218951    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511360df-9654-4ab9-a5ba-70bdb1512d21" containerName="manager"
Jul 24 14:54:32.218982 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.218965    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="511360df-9654-4ab9-a5ba-70bdb1512d21" containerName="manager"
Jul 24 14:54:32.219059 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.219019    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="511360df-9654-4ab9-a5ba-70bdb1512d21" containerName="manager"
Jul 24 14:54:32.222261 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.222245    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:54:32.224606 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.224585    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-knv64\""
Jul 24 14:54:32.233082 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.233062    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"]
Jul 24 14:54:32.242676 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.242655    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfppb\" (UniqueName: \"kubernetes.io/projected/4e243009-02ee-4f91-aa17-36a85a86dd93-kube-api-access-vfppb\") pod \"kuadrant-operator-controller-manager-86878c769f-lhp6j\" (UID: \"4e243009-02ee-4f91-aa17-36a85a86dd93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:54:32.242796 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.242696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e243009-02ee-4f91-aa17-36a85a86dd93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lhp6j\" (UID: \"4e243009-02ee-4f91-aa17-36a85a86dd93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:54:32.343236 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.343209    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e243009-02ee-4f91-aa17-36a85a86dd93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lhp6j\" (UID: \"4e243009-02ee-4f91-aa17-36a85a86dd93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:54:32.343341 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.343317    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfppb\" (UniqueName: \"kubernetes.io/projected/4e243009-02ee-4f91-aa17-36a85a86dd93-kube-api-access-vfppb\") pod \"kuadrant-operator-controller-manager-86878c769f-lhp6j\" (UID: \"4e243009-02ee-4f91-aa17-36a85a86dd93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:54:32.343541 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.343521    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e243009-02ee-4f91-aa17-36a85a86dd93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lhp6j\" (UID: \"4e243009-02ee-4f91-aa17-36a85a86dd93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:54:32.351497 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.351480    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfppb\" (UniqueName: \"kubernetes.io/projected/4e243009-02ee-4f91-aa17-36a85a86dd93-kube-api-access-vfppb\") pod \"kuadrant-operator-controller-manager-86878c769f-lhp6j\" (UID: \"4e243009-02ee-4f91-aa17-36a85a86dd93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:54:32.531442 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.531394    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:54:32.664406 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:32.664378    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"]
Jul 24 14:54:32.667735 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:54:32.667703    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e243009_02ee_4f91_aa17_36a85a86dd93.slice/crio-906e28032d3c3a04fa94177e7d0d2c78f26cb6b46cf597bb476c4ac67a0470c7 WatchSource:0}: Error finding container 906e28032d3c3a04fa94177e7d0d2c78f26cb6b46cf597bb476c4ac67a0470c7: Status 404 returned error can't find the container with id 906e28032d3c3a04fa94177e7d0d2c78f26cb6b46cf597bb476c4ac67a0470c7
Jul 24 14:54:33.382287 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:33.382245    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j" event={"ID":"4e243009-02ee-4f91-aa17-36a85a86dd93","Type":"ContainerStarted","Data":"3151c37806ba280d413eeeb39e3b42c4779a2fe90c27c78bebe2c5239618b24d"}
Jul 24 14:54:33.382287 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:33.382289    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j" event={"ID":"4e243009-02ee-4f91-aa17-36a85a86dd93","Type":"ContainerStarted","Data":"906e28032d3c3a04fa94177e7d0d2c78f26cb6b46cf597bb476c4ac67a0470c7"}
Jul 24 14:54:33.383019 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:33.382383    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:54:33.407351 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:33.407301    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j" podStartSLOduration=1.407288219 podStartE2EDuration="1.407288219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:54:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:54:33.405004976 +0000 UTC m=+1258.558005700" watchObservedRunningTime="2026-07-24 14:54:33.407288219 +0000 UTC m=+1258.560288943"
Jul 24 14:54:36.728318 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:36.728289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:54:44.387764 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:54:44.387731    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lhp6j"
Jul 24 14:56:33.121282 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:56:33.121248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:57:06.910867 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:06.910834    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:57:17.442097 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.442065    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"]
Jul 24 14:57:17.445384 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.445367    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.448093 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.448069    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-38e4a577-kserve-self-signed-certs\""
Jul 24 14:57:17.482232 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.482209    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"]
Jul 24 14:57:17.490040 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.490016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:57:17.582141 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.582116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tmp-dir\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.582290 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.582163    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-dshm\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.582290 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.582243    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bffr7\" (UniqueName: \"kubernetes.io/projected/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-kube-api-access-bffr7\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.582412 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.582290    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-model-cache\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.582412 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.582317    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tls-certs\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.582412 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.582355    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-home\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.682757 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.682726    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-dshm\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.682875 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.682773    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bffr7\" (UniqueName: \"kubernetes.io/projected/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-kube-api-access-bffr7\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.682875 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.682793    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-model-cache\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.682875 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.682809    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tls-certs\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.682875 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.682830    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-home\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.682875 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.682868    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tmp-dir\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.683183 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.683160    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-model-cache\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.683251 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.683232    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tmp-dir\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.683287 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.683261    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-home\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.684752 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.684723    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-dshm\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.685079 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.685062    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tls-certs\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.695733 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.695683    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bffr7\" (UniqueName: \"kubernetes.io/projected/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-kube-api-access-bffr7\") pod \"e2e-same-id-a-38e4a577-kserve-78598f7647-9b258\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") " pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.754308 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.754287    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:17.890449 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:17.889105    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"]
Jul 24 14:57:17.892153 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:57:17.892128    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91412ebb_4d26_4d5f_b009_37f7ac8e4a3a.slice/crio-9000a0bf91df9e9ddcd4e3581a38362a11c84aa77e9fc8d8876739fb4616b4a1 WatchSource:0}: Error finding container 9000a0bf91df9e9ddcd4e3581a38362a11c84aa77e9fc8d8876739fb4616b4a1: Status 404 returned error can't find the container with id 9000a0bf91df9e9ddcd4e3581a38362a11c84aa77e9fc8d8876739fb4616b4a1
Jul 24 14:57:18.118225 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:18.118191    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258" event={"ID":"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a","Type":"ContainerStarted","Data":"978b042b0fc80f0bb5e0a4d3abc6a6908374e44a5ddf46ddb976304a9a9fad14"}
Jul 24 14:57:18.118359 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:18.118234    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258" event={"ID":"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a","Type":"ContainerStarted","Data":"9000a0bf91df9e9ddcd4e3581a38362a11c84aa77e9fc8d8876739fb4616b4a1"}
Jul 24 14:57:18.140512 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:18.140443    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258" podStartSLOduration=1.140425629 podStartE2EDuration="1.140425629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:57:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:18.139425317 +0000 UTC m=+1423.292426041" watchObservedRunningTime="2026-07-24 14:57:18.140425629 +0000 UTC m=+1423.293426350"
Jul 24 14:57:19.594266 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.594232    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"]
Jul 24 14:57:19.597267 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.597244    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.600478 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.600439    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-38e4a577-kserve-self-signed-certs\""
Jul 24 14:57:19.613980 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.613956    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"]
Jul 24 14:57:19.647261 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.647230    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:57:19.699173 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.699151    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtth8\" (UniqueName: \"kubernetes.io/projected/d458e684-005a-4c41-93b5-b28bf8b1d02f-kube-api-access-mtth8\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.699277 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.699210    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d458e684-005a-4c41-93b5-b28bf8b1d02f-tls-certs\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.699338 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.699268    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-tmp-dir\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.699338 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.699297    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-dshm\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.699444 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.699337    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-model-cache\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.699444 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.699396    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-home\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.800688 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.800658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-home\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.800833 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.800710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtth8\" (UniqueName: \"kubernetes.io/projected/d458e684-005a-4c41-93b5-b28bf8b1d02f-kube-api-access-mtth8\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.800904 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.800839    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d458e684-005a-4c41-93b5-b28bf8b1d02f-tls-certs\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.800904 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.800892    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-tmp-dir\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.801006 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.800935    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-dshm\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.801006 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.800973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-model-cache\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.801101 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.800995    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-home\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.801215 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.801175    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-tmp-dir\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.801314 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.801294    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-model-cache\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.803051 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.803031    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-dshm\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.803122 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.803108    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d458e684-005a-4c41-93b5-b28bf8b1d02f-tls-certs\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.810600 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.810574    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtth8\" (UniqueName: \"kubernetes.io/projected/d458e684-005a-4c41-93b5-b28bf8b1d02f-kube-api-access-mtth8\") pod \"e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") " pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:19.906848 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:19.906797    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:20.035838 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:20.035811    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"]
Jul 24 14:57:20.038617 ip-10-0-141-133 kubenswrapper[2583]: W0724 14:57:20.038587    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd458e684_005a_4c41_93b5_b28bf8b1d02f.slice/crio-5c3efb090cd0a3a85e2f5b8d169e736841e8a0de80b2b1b0f1abbe5ce4e33734 WatchSource:0}: Error finding container 5c3efb090cd0a3a85e2f5b8d169e736841e8a0de80b2b1b0f1abbe5ce4e33734: Status 404 returned error can't find the container with id 5c3efb090cd0a3a85e2f5b8d169e736841e8a0de80b2b1b0f1abbe5ce4e33734
Jul 24 14:57:20.128364 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:20.128327    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm" event={"ID":"d458e684-005a-4c41-93b5-b28bf8b1d02f","Type":"ContainerStarted","Data":"ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239"}
Jul 24 14:57:20.128510 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:20.128372    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm" event={"ID":"d458e684-005a-4c41-93b5-b28bf8b1d02f","Type":"ContainerStarted","Data":"5c3efb090cd0a3a85e2f5b8d169e736841e8a0de80b2b1b0f1abbe5ce4e33734"}
Jul 24 14:57:20.155060 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:20.155020    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm" podStartSLOduration=1.154998355 podStartE2EDuration="1.154998355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:20.152239608 +0000 UTC m=+1425.305240333" watchObservedRunningTime="2026-07-24 14:57:20.154998355 +0000 UTC m=+1425.307999079"
Jul 24 14:57:27.754784 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:27.754755    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:27.755207 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:27.754870    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:27.766759 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:27.766729    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:28.176374 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:28.176350    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:29.907472 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:29.907424    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:29.907822 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:29.907495    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:29.919029 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:29.919003    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:30.184900 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:30.184839    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:57:44.146202 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:44.146170    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"]
Jul 24 14:57:44.259569 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:44.259534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:57:44.955702 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:44.955667    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"]
Jul 24 14:57:44.970722 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:44.970697    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 14:57:59.163912 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.163872    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258" podUID="91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" containerName="main" containerID="cri-o://978b042b0fc80f0bb5e0a4d3abc6a6908374e44a5ddf46ddb976304a9a9fad14" gracePeriod=45
Jul 24 14:57:59.308371 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.308344    2583 generic.go:356] "Generic (PLEG): container finished" podID="91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" containerID="978b042b0fc80f0bb5e0a4d3abc6a6908374e44a5ddf46ddb976304a9a9fad14" exitCode=0
Jul 24 14:57:59.308520 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.308417    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258" event={"ID":"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a","Type":"ContainerDied","Data":"978b042b0fc80f0bb5e0a4d3abc6a6908374e44a5ddf46ddb976304a9a9fad14"}
Jul 24 14:57:59.408782 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.408761    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:57:59.517839 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.517773    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-model-cache\") pod \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") "
Jul 24 14:57:59.517839 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.517822    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tmp-dir\") pod \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") "
Jul 24 14:57:59.517839 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.517846    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bffr7\" (UniqueName: \"kubernetes.io/projected/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-kube-api-access-bffr7\") pod \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") "
Jul 24 14:57:59.518068 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.517864    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tls-certs\") pod \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") "
Jul 24 14:57:59.518068 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.517900    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-home\") pod \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") "
Jul 24 14:57:59.518068 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.517954    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-dshm\") pod \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\" (UID: \"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a\") "
Jul 24 14:57:59.518068 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.518013    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" (UID: "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:57:59.518068 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.518031    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-model-cache" (OuterVolumeSpecName: "model-cache") pod "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" (UID: "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:57:59.518246 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.518197    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-home" (OuterVolumeSpecName: "home") pod "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" (UID: "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:57:59.518246 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.518225    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-model-cache\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:57:59.518246 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.518240    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tmp-dir\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:57:59.519985 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.519954    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-kube-api-access-bffr7" (OuterVolumeSpecName: "kube-api-access-bffr7") pod "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" (UID: "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a"). InnerVolumeSpecName "kube-api-access-bffr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:57:59.519985 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.519969    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" (UID: "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:57:59.519985 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.519972    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-dshm" (OuterVolumeSpecName: "dshm") pod "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" (UID: "91412ebb-4d26-4d5f-b009-37f7ac8e4a3a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:57:59.619133 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.619114    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-dshm\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:57:59.619133 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.619133    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bffr7\" (UniqueName: \"kubernetes.io/projected/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-kube-api-access-bffr7\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:57:59.619262 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.619142    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-tls-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:57:59.619262 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.619151    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a-home\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:57:59.973318 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:57:59.973282    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm" podUID="d458e684-005a-4c41-93b5-b28bf8b1d02f" containerName="main" containerID="cri-o://ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239" gracePeriod=45
Jul 24 14:58:00.236282 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.236232    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:58:00.314415 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.314390    2583 generic.go:356] "Generic (PLEG): container finished" podID="d458e684-005a-4c41-93b5-b28bf8b1d02f" containerID="ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239" exitCode=0
Jul 24 14:58:00.314546 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.314428    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"
Jul 24 14:58:00.314546 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.314478    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm" event={"ID":"d458e684-005a-4c41-93b5-b28bf8b1d02f","Type":"ContainerDied","Data":"ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239"}
Jul 24 14:58:00.314546 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.314522    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm" event={"ID":"d458e684-005a-4c41-93b5-b28bf8b1d02f","Type":"ContainerDied","Data":"5c3efb090cd0a3a85e2f5b8d169e736841e8a0de80b2b1b0f1abbe5ce4e33734"}
Jul 24 14:58:00.314546 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.314543    2583 scope.go:117] "RemoveContainer" containerID="ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239"
Jul 24 14:58:00.315617 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.315597    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258" event={"ID":"91412ebb-4d26-4d5f-b009-37f7ac8e4a3a","Type":"ContainerDied","Data":"9000a0bf91df9e9ddcd4e3581a38362a11c84aa77e9fc8d8876739fb4616b4a1"}
Jul 24 14:58:00.315680 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.315655    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"
Jul 24 14:58:00.323566 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.323544    2583 scope.go:117] "RemoveContainer" containerID="ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239"
Jul 24 14:58:00.323807 ip-10-0-141-133 kubenswrapper[2583]: E0724 14:58:00.323785    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239\": container with ID starting with ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239 not found: ID does not exist" containerID="ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239"
Jul 24 14:58:00.323889 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.323816    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239"} err="failed to get container status \"ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239\": rpc error: code = NotFound desc = could not find container \"ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239\": container with ID starting with ea7e2380f8c40e2f786b3d6f6a8571fce8f0545e8c6a7eed80449392404cb239 not found: ID does not exist"
Jul 24 14:58:00.323889 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.323838    2583 scope.go:117] "RemoveContainer" containerID="978b042b0fc80f0bb5e0a4d3abc6a6908374e44a5ddf46ddb976304a9a9fad14"
Jul 24 14:58:00.325073 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325056    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-home\") pod \"d458e684-005a-4c41-93b5-b28bf8b1d02f\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") "
Jul 24 14:58:00.325140 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325099    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-tmp-dir\") pod \"d458e684-005a-4c41-93b5-b28bf8b1d02f\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") "
Jul 24 14:58:00.325177 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325140    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtth8\" (UniqueName: \"kubernetes.io/projected/d458e684-005a-4c41-93b5-b28bf8b1d02f-kube-api-access-mtth8\") pod \"d458e684-005a-4c41-93b5-b28bf8b1d02f\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") "
Jul 24 14:58:00.325177 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325167    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-model-cache\") pod \"d458e684-005a-4c41-93b5-b28bf8b1d02f\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") "
Jul 24 14:58:00.325269 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325228    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-dshm\") pod \"d458e684-005a-4c41-93b5-b28bf8b1d02f\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") "
Jul 24 14:58:00.325308 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325289    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d458e684-005a-4c41-93b5-b28bf8b1d02f-tls-certs\") pod \"d458e684-005a-4c41-93b5-b28bf8b1d02f\" (UID: \"d458e684-005a-4c41-93b5-b28bf8b1d02f\") "
Jul 24 14:58:00.325366 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325342    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-home" (OuterVolumeSpecName: "home") pod "d458e684-005a-4c41-93b5-b28bf8b1d02f" (UID: "d458e684-005a-4c41-93b5-b28bf8b1d02f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:58:00.325435 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325387    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d458e684-005a-4c41-93b5-b28bf8b1d02f" (UID: "d458e684-005a-4c41-93b5-b28bf8b1d02f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:58:00.325516 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325441    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-model-cache" (OuterVolumeSpecName: "model-cache") pod "d458e684-005a-4c41-93b5-b28bf8b1d02f" (UID: "d458e684-005a-4c41-93b5-b28bf8b1d02f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:58:00.325619 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325593    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-home\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:58:00.325692 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325624    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-tmp-dir\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:58:00.325692 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.325639    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-model-cache\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:58:00.327147 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.327118    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d458e684-005a-4c41-93b5-b28bf8b1d02f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d458e684-005a-4c41-93b5-b28bf8b1d02f" (UID: "d458e684-005a-4c41-93b5-b28bf8b1d02f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:58:00.327223 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.327140    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-dshm" (OuterVolumeSpecName: "dshm") pod "d458e684-005a-4c41-93b5-b28bf8b1d02f" (UID: "d458e684-005a-4c41-93b5-b28bf8b1d02f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:58:00.327223 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.327193    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d458e684-005a-4c41-93b5-b28bf8b1d02f-kube-api-access-mtth8" (OuterVolumeSpecName: "kube-api-access-mtth8") pod "d458e684-005a-4c41-93b5-b28bf8b1d02f" (UID: "d458e684-005a-4c41-93b5-b28bf8b1d02f"). InnerVolumeSpecName "kube-api-access-mtth8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:58:00.353012 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.352976    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"]
Jul 24 14:58:00.355438 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.355417    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-38e4a577-kserve-78598f7647-9b258"]
Jul 24 14:58:00.427000 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.426981    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtth8\" (UniqueName: \"kubernetes.io/projected/d458e684-005a-4c41-93b5-b28bf8b1d02f-kube-api-access-mtth8\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:58:00.427000 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.426999    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d458e684-005a-4c41-93b5-b28bf8b1d02f-dshm\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:58:00.427112 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.427012    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d458e684-005a-4c41-93b5-b28bf8b1d02f-tls-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 14:58:00.638300 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.638276    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"]
Jul 24 14:58:00.650163 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:00.650140    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm"]
Jul 24 14:58:01.342029 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:01.341999    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" path="/var/lib/kubelet/pods/91412ebb-4d26-4d5f-b009-37f7ac8e4a3a/volumes"
Jul 24 14:58:01.342351 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:01.342343    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d458e684-005a-4c41-93b5-b28bf8b1d02f" path="/var/lib/kubelet/pods/d458e684-005a-4c41-93b5-b28bf8b1d02f/volumes"
Jul 24 14:58:05.175390 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:58:05.175351    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-38e4a577-kserve-f75454f87-qlktm" podUID="d458e684-005a-4c41-93b5-b28bf8b1d02f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 24 14:59:22.053095 ip-10-0-141-133 kubenswrapper[2583]: I0724 14:59:22.053006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:56.259516 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:00:56.259462    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:18.727576 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:18.727542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:22.012198 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:22.012159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:22.129337 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:22.129295    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:22.685174 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:22.685140    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:29.974292 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:29.974257    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:30.146129 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.146094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:30.438705 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.438670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:30.663723 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.663684    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"]
Jul 24 15:01:30.664150 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.664130    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" containerName="main"
Jul 24 15:01:30.664243 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.664152    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" containerName="main"
Jul 24 15:01:30.664243 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.664170    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d458e684-005a-4c41-93b5-b28bf8b1d02f" containerName="main"
Jul 24 15:01:30.664243 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.664178    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d458e684-005a-4c41-93b5-b28bf8b1d02f" containerName="main"
Jul 24 15:01:30.664387 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.664261    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="91412ebb-4d26-4d5f-b009-37f7ac8e4a3a" containerName="main"
Jul 24 15:01:30.664387 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.664275    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d458e684-005a-4c41-93b5-b28bf8b1d02f" containerName="main"
Jul 24 15:01:30.667539 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.667517    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.672960 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.672940    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ee2b828d-openshift-default-dockercfg-4h2ck\""
Jul 24 15:01:30.726379 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.726282    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"]
Jul 24 15:01:30.739771 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.739731    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:30.749856 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.749827    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c508f91-622a-4550-8a49-8c24cba87b1e-istiod-ca-cert\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.749992 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.749860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml26p\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-kube-api-access-ml26p\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.749992 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.749887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-socket\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.750072 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.749999    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-data\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.750072 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.750052    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-certs\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.750141 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.750084    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-credential-socket\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.750141 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.750131    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-token\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.750216 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.750167    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-podinfo\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.750253 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.750223    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-envoy\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.851658 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.851626    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-podinfo\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.851844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.851679    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-envoy\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.851844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.851706    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c508f91-622a-4550-8a49-8c24cba87b1e-istiod-ca-cert\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.851844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.851724    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml26p\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-kube-api-access-ml26p\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.851844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.851751    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-socket\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.851844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.851789    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-data\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.851844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.851829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-certs\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.852133 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.851854    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-credential-socket\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.852133 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.851878    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-token\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.852378 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.852349    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-credential-socket\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.852540 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.852390    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-certs\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.852540 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.852437    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-data\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.852643 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.852540    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-socket\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.852643 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.852588    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c508f91-622a-4550-8a49-8c24cba87b1e-istiod-ca-cert\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.854079 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.854054    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-envoy\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.854200 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.854180    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-podinfo\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.879555 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.879524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml26p\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-kube-api-access-ml26p\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.893243 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.893207    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-token\") pod \"e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") " pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:30.977600 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:30.977513    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:31.011174 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:31.011140    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:31.139968 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:31.139930    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"]
Jul 24 15:01:31.146885 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:01:31.146859    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c508f91_622a_4550_8a49_8c24cba87b1e.slice/crio-607cfec99dc0cdd9ae948989b419547a86791e19196820729cde4c9133d92bb1 WatchSource:0}: Error finding container 607cfec99dc0cdd9ae948989b419547a86791e19196820729cde4c9133d92bb1: Status 404 returned error can't find the container with id 607cfec99dc0cdd9ae948989b419547a86791e19196820729cde4c9133d92bb1
Jul 24 15:01:31.149023 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:31.148989    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:01:31.149100 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:31.149063    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:01:31.149100 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:31.149091    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:01:31.296384 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:31.296296    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" event={"ID":"3c508f91-622a-4550-8a49-8c24cba87b1e","Type":"ContainerStarted","Data":"57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce"}
Jul 24 15:01:31.296384 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:31.296357    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" event={"ID":"3c508f91-622a-4550-8a49-8c24cba87b1e","Type":"ContainerStarted","Data":"607cfec99dc0cdd9ae948989b419547a86791e19196820729cde4c9133d92bb1"}
Jul 24 15:01:31.977612 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:31.977579    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:32.799623 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:32.799587    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:32.979585 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:32.979522    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" podStartSLOduration=2.979500273 podStartE2EDuration="2.979500273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:31.343866513 +0000 UTC m=+1676.496867239" watchObservedRunningTime="2026-07-24 15:01:32.979500273 +0000 UTC m=+1678.132501003"
Jul 24 15:01:32.980423 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:32.980399    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"]
Jul 24 15:01:32.983887 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:32.983867    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:32.984054 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:32.984035    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:32.993662 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:32.993638    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ee2b828d-serving-cert\""
Jul 24 15:01:33.006639 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.006609    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"]
Jul 24 15:01:33.057589 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.057283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:33.069692 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.069665    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-maas-api-tls\") pod \"maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.069830 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.069701    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gt47\" (UniqueName: \"kubernetes.io/projected/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-kube-api-access-5gt47\") pod \"maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.069830 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.069792    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-openshift-service-ca\") pod \"maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.170242 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.170201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-maas-api-tls\") pod \"maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.170242 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.170243    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gt47\" (UniqueName: \"kubernetes.io/projected/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-kube-api-access-5gt47\") pod \"maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.170513 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.170295    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-openshift-service-ca\") pod \"maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.170918 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.170892    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-openshift-service-ca\") pod \"maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.173031 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.172995    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-maas-api-tls\") pod \"maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.184814 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.184772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:33.186865 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.186843    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gt47\" (UniqueName: \"kubernetes.io/projected/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-kube-api-access-5gt47\") pod \"maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.294540 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.294507    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:33.306329 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.306297    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:33.307421 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.307400    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:33.662893 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.662867    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"]
Jul 24 15:01:33.665531 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:01:33.665504    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaeaa9e0_a0d6_4dc9_b7fe_4d1ae137c431.slice/crio-75eda2dae5dadecc4699732bc7741af021b93bb81bde8d06c2cc53de5a908417 WatchSource:0}: Error finding container 75eda2dae5dadecc4699732bc7741af021b93bb81bde8d06c2cc53de5a908417: Status 404 returned error can't find the container with id 75eda2dae5dadecc4699732bc7741af021b93bb81bde8d06c2cc53de5a908417
Jul 24 15:01:33.666915 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.666896    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:01:33.805280 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.805249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:33.921990 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:33.921899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:34.013750 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.013709    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"]
Jul 24 15:01:34.018300 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.018280    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:34.021223 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.021197    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ee2b828d\""
Jul 24 15:01:34.021348 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.021242    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ee2b828d-dockercfg-zncpz\""
Jul 24 15:01:34.030402 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.030372    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"]
Jul 24 15:01:34.079041 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.079009    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkp26\" (UniqueName: \"kubernetes.io/projected/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-kube-api-access-jkp26\") pod \"payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb\" (UID: \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:34.079250 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.079063    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb\" (UID: \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:34.090151 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.090121    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"]
Jul 24 15:01:34.093513 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.093496    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:34.113138 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.113110    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"]
Jul 24 15:01:34.180268 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.180209    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb\" (UID: \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:34.180381 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.180267    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm6sv\" (UniqueName: \"kubernetes.io/projected/6da2612a-e116-4dff-8597-417e67db9406-kube-api-access-pm6sv\") pod \"payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb\" (UID: \"6da2612a-e116-4dff-8597-417e67db9406\") " pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:34.180381 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.180289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da2612a-e116-4dff-8597-417e67db9406-plugins-config-volume\") pod \"payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb\" (UID: \"6da2612a-e116-4dff-8597-417e67db9406\") " pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:34.180381 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.180331    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkp26\" (UniqueName: \"kubernetes.io/projected/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-kube-api-access-jkp26\") pod \"payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb\" (UID: \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:34.180876 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.180852    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb\" (UID: \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:34.189725 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.189703    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkp26\" (UniqueName: \"kubernetes.io/projected/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-kube-api-access-jkp26\") pod \"payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb\" (UID: \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:34.281682 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.281645    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm6sv\" (UniqueName: \"kubernetes.io/projected/6da2612a-e116-4dff-8597-417e67db9406-kube-api-access-pm6sv\") pod \"payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb\" (UID: \"6da2612a-e116-4dff-8597-417e67db9406\") " pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:34.281682 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.281680    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da2612a-e116-4dff-8597-417e67db9406-plugins-config-volume\") pod \"payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb\" (UID: \"6da2612a-e116-4dff-8597-417e67db9406\") " pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:34.282323 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.282303    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da2612a-e116-4dff-8597-417e67db9406-plugins-config-volume\") pod \"payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb\" (UID: \"6da2612a-e116-4dff-8597-417e67db9406\") " pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:34.291110 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.291084    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm6sv\" (UniqueName: \"kubernetes.io/projected/6da2612a-e116-4dff-8597-417e67db9406-kube-api-access-pm6sv\") pod \"payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb\" (UID: \"6da2612a-e116-4dff-8597-417e67db9406\") " pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:34.312067 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.312019    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l" event={"ID":"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431","Type":"ContainerStarted","Data":"b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72"}
Jul 24 15:01:34.312314 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.312094    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l" event={"ID":"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431","Type":"ContainerStarted","Data":"75eda2dae5dadecc4699732bc7741af021b93bb81bde8d06c2cc53de5a908417"}
Jul 24 15:01:34.329680 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.329651    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:34.336067 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.336003    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l" podStartSLOduration=1.851541885 podStartE2EDuration="2.335984903s" totalImagesPullingTime="484.443018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:34.333678771 +0000 UTC m=+1679.486679499" watchObservedRunningTime="2026-07-24 15:01:34.335984903 +0000 UTC m=+1679.488985629"
Jul 24 15:01:34.403858 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.403814    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:34.484825 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.484793    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"]
Jul 24 15:01:34.486275 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:01:34.486242    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5e7dab_54ad_4cd1_a48e_5bfc77c021c8.slice/crio-c524ef6bb3b1cef6378379535bbef34d29d3388834393d634d2f8709f06bbc15 WatchSource:0}: Error finding container c524ef6bb3b1cef6378379535bbef34d29d3388834393d634d2f8709f06bbc15: Status 404 returned error can't find the container with id c524ef6bb3b1cef6378379535bbef34d29d3388834393d634d2f8709f06bbc15
Jul 24 15:01:34.566876 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:34.566848    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"]
Jul 24 15:01:34.569717 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:01:34.569689    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da2612a_e116_4dff_8597_417e67db9406.slice/crio-a61fdcc9c1c7b49d53a3c1864026adf91ecfacdbd45f955b95106b54d9a0c951 WatchSource:0}: Error finding container a61fdcc9c1c7b49d53a3c1864026adf91ecfacdbd45f955b95106b54d9a0c951: Status 404 returned error can't find the container with id a61fdcc9c1c7b49d53a3c1864026adf91ecfacdbd45f955b95106b54d9a0c951
Jul 24 15:01:35.318341 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:35.318301    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb" event={"ID":"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8","Type":"ContainerStarted","Data":"f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46"}
Jul 24 15:01:35.318776 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:35.318367    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:35.318776 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:35.318395    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb" event={"ID":"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8","Type":"ContainerStarted","Data":"c524ef6bb3b1cef6378379535bbef34d29d3388834393d634d2f8709f06bbc15"}
Jul 24 15:01:35.319781 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:35.319753    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb" event={"ID":"6da2612a-e116-4dff-8597-417e67db9406","Type":"ContainerStarted","Data":"0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2"}
Jul 24 15:01:35.319926 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:35.319795    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb" event={"ID":"6da2612a-e116-4dff-8597-417e67db9406","Type":"ContainerStarted","Data":"a61fdcc9c1c7b49d53a3c1864026adf91ecfacdbd45f955b95106b54d9a0c951"}
Jul 24 15:01:35.319926 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:35.319897    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:35.371970 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:35.371918    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb" podStartSLOduration=2.37190323 podStartE2EDuration="2.37190323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:35.341013392 +0000 UTC m=+1680.494014120" watchObservedRunningTime="2026-07-24 15:01:35.37190323 +0000 UTC m=+1680.524903964"
Jul 24 15:01:35.372193 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:35.372175    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb" podStartSLOduration=1.372168867 podStartE2EDuration="1.372168867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:35.370589508 +0000 UTC m=+1680.523590234" watchObservedRunningTime="2026-07-24 15:01:35.372168867 +0000 UTC m=+1680.525169596"
Jul 24 15:01:36.324576 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:36.324540    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:41.329148 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:41.329120    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:01:46.326975 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:46.326930    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:01:47.331651 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:47.331621    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:01:48.307841 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:48.307806    2583 patch_prober.go:28] interesting pod/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 24 15:01:48.308000 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:48.307865    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 24 15:01:48.387940 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:48.387881    2583 generic.go:356] "Generic (PLEG): container finished" podID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerID="57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce" exitCode=0
Jul 24 15:01:48.387940 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:48.387935    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" event={"ID":"3c508f91-622a-4550-8a49-8c24cba87b1e","Type":"ContainerDied","Data":"57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce"}
Jul 24 15:01:48.388304 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:48.388290    2583 scope.go:117] "RemoveContainer" containerID="57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce"
Jul 24 15:01:48.389335 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:48.389310    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:01:48.389403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:48.389378    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:01:48.389436 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:48.389410    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:01:49.394814 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:49.394784    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" event={"ID":"3c508f91-622a-4550-8a49-8c24cba87b1e","Type":"ContainerStarted","Data":"b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4"}
Jul 24 15:01:49.978640 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:49.978608    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:49.978807 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:49.978746    2583 patch_prober.go:28] interesting pod/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 24 15:01:49.978807 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:49.978793    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 24 15:01:50.977820 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:50.977786    2583 patch_prober.go:28] interesting pod/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 24 15:01:50.978202 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:50.977861    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 24 15:01:51.978161 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:51.978122    2583 patch_prober.go:28] interesting pod/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 24 15:01:51.978534 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:51.978179    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 24 15:01:52.978048 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:52.978011    2583 patch_prober.go:28] interesting pod/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 24 15:01:52.978209 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:52.978067    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 24 15:01:53.981008 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:53.980976    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:53.981354 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:53.981315    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:01:53.982014 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:01:53.981994    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:02:08.865726 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:08.865524    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"]
Jul 24 15:02:08.866429 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:08.866384    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l" podUID="aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431" containerName="maas-api" containerID="cri-o://b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72" gracePeriod=30
Jul 24 15:02:08.897113 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:08.897087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:08.929849 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:08.929825    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"]
Jul 24 15:02:08.930073 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:08.930047    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb" podUID="6da2612a-e116-4dff-8597-417e67db9406" containerName="payload-processing" containerID="cri-o://0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2" gracePeriod=30
Jul 24 15:02:08.942345 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:08.941812    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"]
Jul 24 15:02:08.942345 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:08.942009    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb" podUID="de5e7dab-54ad-4cd1-a48e-5bfc77c021c8" containerName="payload-pre-processing" containerID="cri-o://f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46" gracePeriod=30
Jul 24 15:02:08.977967 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:08.977938    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:09.202750 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.202729    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:02:09.206117 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.206091    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:02:09.211364 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.211330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:09.250159 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.250139    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:02:09.353585 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.353561    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gt47\" (UniqueName: \"kubernetes.io/projected/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-kube-api-access-5gt47\") pod \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") "
Jul 24 15:02:09.353728 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.353597    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da2612a-e116-4dff-8597-417e67db9406-plugins-config-volume\") pod \"6da2612a-e116-4dff-8597-417e67db9406\" (UID: \"6da2612a-e116-4dff-8597-417e67db9406\") "
Jul 24 15:02:09.353728 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.353628    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-openshift-service-ca\") pod \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") "
Jul 24 15:02:09.353728 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.353658    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkp26\" (UniqueName: \"kubernetes.io/projected/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-kube-api-access-jkp26\") pod \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\" (UID: \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\") "
Jul 24 15:02:09.353728 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.353677    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-plugins-config-volume\") pod \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\" (UID: \"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8\") "
Jul 24 15:02:09.353728 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.353693    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-maas-api-tls\") pod \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\" (UID: \"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431\") "
Jul 24 15:02:09.354001 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.353785    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm6sv\" (UniqueName: \"kubernetes.io/projected/6da2612a-e116-4dff-8597-417e67db9406-kube-api-access-pm6sv\") pod \"6da2612a-e116-4dff-8597-417e67db9406\" (UID: \"6da2612a-e116-4dff-8597-417e67db9406\") "
Jul 24 15:02:09.354001 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.353944    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6da2612a-e116-4dff-8597-417e67db9406-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6da2612a-e116-4dff-8597-417e67db9406" (UID: "6da2612a-e116-4dff-8597-417e67db9406"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:09.354093 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.354036    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da2612a-e116-4dff-8597-417e67db9406-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:09.354093 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.354049    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431" (UID: "aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:09.354431 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.354403    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de5e7dab-54ad-4cd1-a48e-5bfc77c021c8" (UID: "de5e7dab-54ad-4cd1-a48e-5bfc77c021c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:09.355762 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.355737    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-kube-api-access-jkp26" (OuterVolumeSpecName: "kube-api-access-jkp26") pod "de5e7dab-54ad-4cd1-a48e-5bfc77c021c8" (UID: "de5e7dab-54ad-4cd1-a48e-5bfc77c021c8"). InnerVolumeSpecName "kube-api-access-jkp26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:09.355844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.355776    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6da2612a-e116-4dff-8597-417e67db9406-kube-api-access-pm6sv" (OuterVolumeSpecName: "kube-api-access-pm6sv") pod "6da2612a-e116-4dff-8597-417e67db9406" (UID: "6da2612a-e116-4dff-8597-417e67db9406"). InnerVolumeSpecName "kube-api-access-pm6sv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:09.356173 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.356140    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431" (UID: "aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:02:09.356173 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.356146    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-kube-api-access-5gt47" (OuterVolumeSpecName: "kube-api-access-5gt47") pod "aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431" (UID: "aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431"). InnerVolumeSpecName "kube-api-access-5gt47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:09.455273 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.455224    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm6sv\" (UniqueName: \"kubernetes.io/projected/6da2612a-e116-4dff-8597-417e67db9406-kube-api-access-pm6sv\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:09.455273 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.455246    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gt47\" (UniqueName: \"kubernetes.io/projected/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-kube-api-access-5gt47\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:09.455273 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.455257    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-openshift-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:09.455273 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.455265    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkp26\" (UniqueName: \"kubernetes.io/projected/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-kube-api-access-jkp26\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:09.455273 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.455273    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:09.455523 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.455282    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431-maas-api-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:09.485417 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.485394    2583 generic.go:356] "Generic (PLEG): container finished" podID="6da2612a-e116-4dff-8597-417e67db9406" containerID="0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2" exitCode=0
Jul 24 15:02:09.485501 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.485438    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"
Jul 24 15:02:09.485501 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.485479    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb" event={"ID":"6da2612a-e116-4dff-8597-417e67db9406","Type":"ContainerDied","Data":"0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2"}
Jul 24 15:02:09.485601 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.485536    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb" event={"ID":"6da2612a-e116-4dff-8597-417e67db9406","Type":"ContainerDied","Data":"a61fdcc9c1c7b49d53a3c1864026adf91ecfacdbd45f955b95106b54d9a0c951"}
Jul 24 15:02:09.485601 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.485558    2583 scope.go:117] "RemoveContainer" containerID="0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2"
Jul 24 15:02:09.486758 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.486742    2583 generic.go:356] "Generic (PLEG): container finished" podID="de5e7dab-54ad-4cd1-a48e-5bfc77c021c8" containerID="f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46" exitCode=0
Jul 24 15:02:09.486829 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.486768    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb" event={"ID":"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8","Type":"ContainerDied","Data":"f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46"}
Jul 24 15:02:09.486829 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.486788    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"
Jul 24 15:02:09.486829 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.486819    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb" event={"ID":"de5e7dab-54ad-4cd1-a48e-5bfc77c021c8","Type":"ContainerDied","Data":"c524ef6bb3b1cef6378379535bbef34d29d3388834393d634d2f8709f06bbc15"}
Jul 24 15:02:09.488373 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.488348    2583 generic.go:356] "Generic (PLEG): container finished" podID="aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431" containerID="b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72" exitCode=0
Jul 24 15:02:09.488516 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.488377    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"
Jul 24 15:02:09.488516 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.488389    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l" event={"ID":"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431","Type":"ContainerDied","Data":"b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72"}
Jul 24 15:02:09.488516 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.488427    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l" event={"ID":"aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431","Type":"ContainerDied","Data":"75eda2dae5dadecc4699732bc7741af021b93bb81bde8d06c2cc53de5a908417"}
Jul 24 15:02:09.495730 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.495709    2583 scope.go:117] "RemoveContainer" containerID="0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2"
Jul 24 15:02:09.496005 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:02:09.495984    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2\": container with ID starting with 0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2 not found: ID does not exist" containerID="0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2"
Jul 24 15:02:09.496066 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.496012    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2"} err="failed to get container status \"0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2\": rpc error: code = NotFound desc = could not find container \"0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2\": container with ID starting with 0dc524be89b1cb4448a15aa69cce2f9fd642bbee303e071cd755d6a0a2d30ad2 not found: ID does not exist"
Jul 24 15:02:09.496066 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.496029    2583 scope.go:117] "RemoveContainer" containerID="f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46"
Jul 24 15:02:09.503675 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.503659    2583 scope.go:117] "RemoveContainer" containerID="f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46"
Jul 24 15:02:09.503907 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:02:09.503891    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46\": container with ID starting with f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46 not found: ID does not exist" containerID="f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46"
Jul 24 15:02:09.503957 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.503914    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46"} err="failed to get container status \"f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46\": rpc error: code = NotFound desc = could not find container \"f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46\": container with ID starting with f320fd2e5bcf00990fb7e22ad00b70f01ebe5b4e6127c2f7d1d6d1b4fcaafc46 not found: ID does not exist"
Jul 24 15:02:09.503957 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.503929    2583 scope.go:117] "RemoveContainer" containerID="b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72"
Jul 24 15:02:09.510523 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.510507    2583 scope.go:117] "RemoveContainer" containerID="b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72"
Jul 24 15:02:09.510758 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:02:09.510741    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72\": container with ID starting with b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72 not found: ID does not exist" containerID="b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72"
Jul 24 15:02:09.510834 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.510765    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72"} err="failed to get container status \"b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72\": rpc error: code = NotFound desc = could not find container \"b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72\": container with ID starting with b345830864f85014c014f7bfed1a55ffb107eb27a7f284917acc52bdc5ebad72 not found: ID does not exist"
Jul 24 15:02:09.518679 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.518658    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"]
Jul 24 15:02:09.527581 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.527560    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb"]
Jul 24 15:02:09.541909 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.541887    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"]
Jul 24 15:02:09.544917 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.544895    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee2b828d-7f9f9867f-jqq4l"]
Jul 24 15:02:09.562047 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.562027    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"]
Jul 24 15:02:09.566971 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:09.566952    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb"]
Jul 24 15:02:11.343887 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:11.343859    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6da2612a-e116-4dff-8597-417e67db9406" path="/var/lib/kubelet/pods/6da2612a-e116-4dff-8597-417e67db9406/volumes"
Jul 24 15:02:11.344310 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:11.344200    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431" path="/var/lib/kubelet/pods/aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431/volumes"
Jul 24 15:02:11.344511 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:11.344497    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de5e7dab-54ad-4cd1-a48e-5bfc77c021c8" path="/var/lib/kubelet/pods/de5e7dab-54ad-4cd1-a48e-5bfc77c021c8/volumes"
Jul 24 15:02:13.550929 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:13.550897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:13.708805 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:13.708771    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"]
Jul 24 15:02:13.709049 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:13.708993    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy" containerID="cri-o://b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4" gracePeriod=30
Jul 24 15:02:13.774130 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:13.774081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:13.989366 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:13.989335    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:14.275743 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:14.275672    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:14.461059 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:14.461010    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:14.607257 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:14.607234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:15.046949 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:15.046861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:15.315403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:15.315323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:17.763902 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.763868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:17.939153 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939122    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"]
Jul 24 15:02:17.939480 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939463    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5e7dab-54ad-4cd1-a48e-5bfc77c021c8" containerName="payload-pre-processing"
Jul 24 15:02:17.939480 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939477    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5e7dab-54ad-4cd1-a48e-5bfc77c021c8" containerName="payload-pre-processing"
Jul 24 15:02:17.939564 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939511    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431" containerName="maas-api"
Jul 24 15:02:17.939564 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939517    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431" containerName="maas-api"
Jul 24 15:02:17.939564 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939524    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6da2612a-e116-4dff-8597-417e67db9406" containerName="payload-processing"
Jul 24 15:02:17.939564 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939531    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6da2612a-e116-4dff-8597-417e67db9406" containerName="payload-processing"
Jul 24 15:02:17.939682 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939581    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaeaa9e0-a0d6-4dc9-b7fe-4d1ae137c431" containerName="maas-api"
Jul 24 15:02:17.939682 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939590    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6da2612a-e116-4dff-8597-417e67db9406" containerName="payload-processing"
Jul 24 15:02:17.939682 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.939599    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="de5e7dab-54ad-4cd1-a48e-5bfc77c021c8" containerName="payload-pre-processing"
Jul 24 15:02:17.944292 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.944271    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:17.951161 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.951138    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-47d0d404-serving-cert\""
Jul 24 15:02:17.960608 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:17.960581    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"]
Jul 24 15:02:18.096257 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.096231    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:18.122185 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.122151    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-openshift-service-ca\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:18.122394 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.122255    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:18.122394 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.122323    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2wv6\" (UniqueName: \"kubernetes.io/projected/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-kube-api-access-n2wv6\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:18.222807 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.222783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:18.222923 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.222833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2wv6\" (UniqueName: \"kubernetes.io/projected/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-kube-api-access-n2wv6\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:18.222923 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.222870    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-openshift-service-ca\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:18.223011 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:02:18.222933    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-serving-cert: secret "maas-api-e2e-migrate-47d0d404-serving-cert" not found
Jul 24 15:02:18.223011 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:02:18.222992    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls podName:15cdd6ea-aef7-4e67-815e-c4f4d9ea2347 nodeName:}" failed. No retries permitted until 2026-07-24 15:02:18.722976103 +0000 UTC m=+1723.875976811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls") pod "maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt" (UID: "15cdd6ea-aef7-4e67-815e-c4f4d9ea2347") : secret "maas-api-e2e-migrate-47d0d404-serving-cert" not found
Jul 24 15:02:18.223516 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.223500    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-openshift-service-ca\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:18.235056 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.235032    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2wv6\" (UniqueName: \"kubernetes.io/projected/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-kube-api-access-n2wv6\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:18.726982 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.726850    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:18.727052 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:02:18.726988    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-serving-cert: secret "maas-api-e2e-migrate-47d0d404-serving-cert" not found
Jul 24 15:02:18.727116 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:02:18.727071    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls podName:15cdd6ea-aef7-4e67-815e-c4f4d9ea2347 nodeName:}" failed. No retries permitted until 2026-07-24 15:02:19.727056113 +0000 UTC m=+1724.880056817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls") pod "maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt" (UID: "15cdd6ea-aef7-4e67-815e-c4f4d9ea2347") : secret "maas-api-e2e-migrate-47d0d404-serving-cert" not found
Jul 24 15:02:18.854412 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.854375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:18.963641 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.963618    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:02:18.998134 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:18.998079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:19.114385 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.114352    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"]
Jul 24 15:02:19.114742 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.114729    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy"
Jul 24 15:02:19.114787 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.114744    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy"
Jul 24 15:02:19.114787 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.114753    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy"
Jul 24 15:02:19.114787 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.114759    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy"
Jul 24 15:02:19.114881 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.114822    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy"
Jul 24 15:02:19.117605 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.117589    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:19.126925 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.126906    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-47d0d404\""
Jul 24 15:02:19.129957 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.129940    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-token\") pod \"3c508f91-622a-4550-8a49-8c24cba87b1e\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") "
Jul 24 15:02:19.130049 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.129979    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-socket\") pod \"3c508f91-622a-4550-8a49-8c24cba87b1e\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") "
Jul 24 15:02:19.130049 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130012    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-envoy\") pod \"3c508f91-622a-4550-8a49-8c24cba87b1e\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") "
Jul 24 15:02:19.130148 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130087    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml26p\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-kube-api-access-ml26p\") pod \"3c508f91-622a-4550-8a49-8c24cba87b1e\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") "
Jul 24 15:02:19.130148 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130118    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-credential-socket\") pod \"3c508f91-622a-4550-8a49-8c24cba87b1e\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") "
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130145    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-podinfo\") pod \"3c508f91-622a-4550-8a49-8c24cba87b1e\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") "
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130281    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c508f91-622a-4550-8a49-8c24cba87b1e-istiod-ca-cert\") pod \"3c508f91-622a-4550-8a49-8c24cba87b1e\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") "
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130331    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-data\") pod \"3c508f91-622a-4550-8a49-8c24cba87b1e\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") "
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130361    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-certs\") pod \"3c508f91-622a-4550-8a49-8c24cba87b1e\" (UID: \"3c508f91-622a-4550-8a49-8c24cba87b1e\") "
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130279    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c508f91-622a-4550-8a49-8c24cba87b1e" (UID: "3c508f91-622a-4550-8a49-8c24cba87b1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130358    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c508f91-622a-4550-8a49-8c24cba87b1e" (UID: "3c508f91-622a-4550-8a49-8c24cba87b1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130597    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c508f91-622a-4550-8a49-8c24cba87b1e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c508f91-622a-4550-8a49-8c24cba87b1e" (UID: "3c508f91-622a-4550-8a49-8c24cba87b1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130709    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130731    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130746    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c508f91-622a-4550-8a49-8c24cba87b1e-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:19.130798 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.130765    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c508f91-622a-4550-8a49-8c24cba87b1e" (UID: "3c508f91-622a-4550-8a49-8c24cba87b1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:19.131349 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.131021    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c508f91-622a-4550-8a49-8c24cba87b1e" (UID: "3c508f91-622a-4550-8a49-8c24cba87b1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:19.133281 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.133250    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c508f91-622a-4550-8a49-8c24cba87b1e" (UID: "3c508f91-622a-4550-8a49-8c24cba87b1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:19.133531 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.133508    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c508f91-622a-4550-8a49-8c24cba87b1e" (UID: "3c508f91-622a-4550-8a49-8c24cba87b1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:19.133664 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.133642    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c508f91-622a-4550-8a49-8c24cba87b1e" (UID: "3c508f91-622a-4550-8a49-8c24cba87b1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:02:19.133982 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.133957    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-kube-api-access-ml26p" (OuterVolumeSpecName: "kube-api-access-ml26p") pod "3c508f91-622a-4550-8a49-8c24cba87b1e" (UID: "3c508f91-622a-4550-8a49-8c24cba87b1e"). InnerVolumeSpecName "kube-api-access-ml26p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:19.148352 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.148331    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"]
Jul 24 15:02:19.214876 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.214838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:19.231782 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.231753    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c9ef36-03c8-4801-8c79-d0c4486a4537-plugins-config-volume\") pod \"payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r\" (UID: \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\") " pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:19.231959 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.231857    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ht9l\" (UniqueName: \"kubernetes.io/projected/f1c9ef36-03c8-4801-8c79-d0c4486a4537-kube-api-access-5ht9l\") pod \"payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r\" (UID: \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\") " pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:19.231959 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.231954    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:19.232072 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.231979    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:19.232072 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.231991    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:19.232072 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.232004    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:19.232072 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.232017    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml26p\" (UniqueName: \"kubernetes.io/projected/3c508f91-622a-4550-8a49-8c24cba87b1e-kube-api-access-ml26p\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:19.232072 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.232028    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c508f91-622a-4550-8a49-8c24cba87b1e-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:19.236695 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.236673    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"]
Jul 24 15:02:19.237072 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.237060    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerName="istio-proxy"
Jul 24 15:02:19.241150 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.241133    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:19.247471 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.247432    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-47d0d404-dockercfg-2pll2\""
Jul 24 15:02:19.266674 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.266626    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"]
Jul 24 15:02:19.332904 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.332860    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ht9l\" (UniqueName: \"kubernetes.io/projected/f1c9ef36-03c8-4801-8c79-d0c4486a4537-kube-api-access-5ht9l\") pod \"payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r\" (UID: \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\") " pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:19.333074 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.333052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c9ef36-03c8-4801-8c79-d0c4486a4537-plugins-config-volume\") pod \"payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r\" (UID: \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\") " pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:19.333953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.333920    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c9ef36-03c8-4801-8c79-d0c4486a4537-plugins-config-volume\") pod \"payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r\" (UID: \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\") " pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:19.354813 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.354788    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ht9l\" (UniqueName: \"kubernetes.io/projected/f1c9ef36-03c8-4801-8c79-d0c4486a4537-kube-api-access-5ht9l\") pod \"payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r\" (UID: \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\") " pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:19.426624 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.426601    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:19.433786 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.433760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b37d2611-eeae-4f8a-a02e-858957a180fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8\" (UID: \"b37d2611-eeae-4f8a-a02e-858957a180fd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:19.433913 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.433886    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgn56\" (UniqueName: \"kubernetes.io/projected/b37d2611-eeae-4f8a-a02e-858957a180fd-kube-api-access-tgn56\") pod \"payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8\" (UID: \"b37d2611-eeae-4f8a-a02e-858957a180fd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:19.534745 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.534665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b37d2611-eeae-4f8a-a02e-858957a180fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8\" (UID: \"b37d2611-eeae-4f8a-a02e-858957a180fd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:19.534882 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.534769    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgn56\" (UniqueName: \"kubernetes.io/projected/b37d2611-eeae-4f8a-a02e-858957a180fd-kube-api-access-tgn56\") pod \"payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8\" (UID: \"b37d2611-eeae-4f8a-a02e-858957a180fd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:19.535740 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.535715    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b37d2611-eeae-4f8a-a02e-858957a180fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8\" (UID: \"b37d2611-eeae-4f8a-a02e-858957a180fd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:19.540415 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.540351    2583 generic.go:356] "Generic (PLEG): container finished" podID="3c508f91-622a-4550-8a49-8c24cba87b1e" containerID="b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4" exitCode=0
Jul 24 15:02:19.540415 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.540390    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" event={"ID":"3c508f91-622a-4550-8a49-8c24cba87b1e","Type":"ContainerDied","Data":"b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4"}
Jul 24 15:02:19.540595 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.540420    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"
Jul 24 15:02:19.540595 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.540479    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz" event={"ID":"3c508f91-622a-4550-8a49-8c24cba87b1e","Type":"ContainerDied","Data":"607cfec99dc0cdd9ae948989b419547a86791e19196820729cde4c9133d92bb1"}
Jul 24 15:02:19.540595 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.540513    2583 scope.go:117] "RemoveContainer" containerID="b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4"
Jul 24 15:02:19.546194 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.546167    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgn56\" (UniqueName: \"kubernetes.io/projected/b37d2611-eeae-4f8a-a02e-858957a180fd-kube-api-access-tgn56\") pod \"payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8\" (UID: \"b37d2611-eeae-4f8a-a02e-858957a180fd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:19.550281 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.550252    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:19.552322 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.552301    2583 scope.go:117] "RemoveContainer" containerID="57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce"
Jul 24 15:02:19.561969 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.561951    2583 scope.go:117] "RemoveContainer" containerID="b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4"
Jul 24 15:02:19.562222 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:02:19.562202    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4\": container with ID starting with b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4 not found: ID does not exist" containerID="b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4"
Jul 24 15:02:19.562266 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.562230    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4"} err="failed to get container status \"b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4\": rpc error: code = NotFound desc = could not find container \"b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4\": container with ID starting with b9f94b8f50633bd65c712c46e33748357afa3641b2bf69f810b7d7f7bfebe8c4 not found: ID does not exist"
Jul 24 15:02:19.562266 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.562246    2583 scope.go:117] "RemoveContainer" containerID="57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce"
Jul 24 15:02:19.562446 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:02:19.562422    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce\": container with ID starting with 57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce not found: ID does not exist" containerID="57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce"
Jul 24 15:02:19.562510 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.562482    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce"} err="failed to get container status \"57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce\": rpc error: code = NotFound desc = could not find container \"57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce\": container with ID starting with 57c106a9b54ee34f223a67f66671e57b0d8ce3c6a52ad0016919c97be6ee75ce not found: ID does not exist"
Jul 24 15:02:19.570404 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.570385    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"]
Jul 24 15:02:19.572662 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:02:19.572642    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c9ef36_03c8_4801_8c79_d0c4486a4537.slice/crio-b801a8a1f4e05c8fec6300ef3fed10071ace4338aed68c6a9c54847b6d86b5d7 WatchSource:0}: Error finding container b801a8a1f4e05c8fec6300ef3fed10071ace4338aed68c6a9c54847b6d86b5d7: Status 404 returned error can't find the container with id b801a8a1f4e05c8fec6300ef3fed10071ace4338aed68c6a9c54847b6d86b5d7
Jul 24 15:02:19.583956 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.583930    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"]
Jul 24 15:02:19.590306 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.590287    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz"]
Jul 24 15:02:19.731815 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.731738    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"]
Jul 24 15:02:19.734272 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:02:19.734242    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb37d2611_eeae_4f8a_a02e_858957a180fd.slice/crio-d0d47d8dfc90fc19cd4afa01a3aedf808fa0244580b8919785858d0903b70ca5 WatchSource:0}: Error finding container d0d47d8dfc90fc19cd4afa01a3aedf808fa0244580b8919785858d0903b70ca5: Status 404 returned error can't find the container with id d0d47d8dfc90fc19cd4afa01a3aedf808fa0244580b8919785858d0903b70ca5
Jul 24 15:02:19.737258 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.737234    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:19.739724 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.739699    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls\") pod \"maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:19.753848 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.753826    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:19.901010 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:19.900979    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"]
Jul 24 15:02:19.904868 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:02:19.904846    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15cdd6ea_aef7_4e67_815e_c4f4d9ea2347.slice/crio-54eafd1af910d1538dfba6c9fd292e2a6c98a6ff9aa988c1fde8313c2c97a823 WatchSource:0}: Error finding container 54eafd1af910d1538dfba6c9fd292e2a6c98a6ff9aa988c1fde8313c2c97a823: Status 404 returned error can't find the container with id 54eafd1af910d1538dfba6c9fd292e2a6c98a6ff9aa988c1fde8313c2c97a823
Jul 24 15:02:20.548562 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:20.548524    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r" event={"ID":"f1c9ef36-03c8-4801-8c79-d0c4486a4537","Type":"ContainerStarted","Data":"7ab26844a4a8ef0ed1cd4f1bf22c8199326f766db5f6f7a523e4aada335e14dd"}
Jul 24 15:02:20.548748 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:20.548597    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:20.548748 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:20.548623    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r" event={"ID":"f1c9ef36-03c8-4801-8c79-d0c4486a4537","Type":"ContainerStarted","Data":"b801a8a1f4e05c8fec6300ef3fed10071ace4338aed68c6a9c54847b6d86b5d7"}
Jul 24 15:02:20.549756 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:20.549722    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt" event={"ID":"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347","Type":"ContainerStarted","Data":"54eafd1af910d1538dfba6c9fd292e2a6c98a6ff9aa988c1fde8313c2c97a823"}
Jul 24 15:02:20.551245 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:20.551218    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8" event={"ID":"b37d2611-eeae-4f8a-a02e-858957a180fd","Type":"ContainerStarted","Data":"ce02ed9f65f1d46f255e2830d9f8b347be6d5f0c476218e53acf5dc6ce043843"}
Jul 24 15:02:20.551357 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:20.551263    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8" event={"ID":"b37d2611-eeae-4f8a-a02e-858957a180fd","Type":"ContainerStarted","Data":"d0d47d8dfc90fc19cd4afa01a3aedf808fa0244580b8919785858d0903b70ca5"}
Jul 24 15:02:20.551357 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:20.551283    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:20.576483 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:20.576425    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r" podStartSLOduration=1.5764123620000001 podStartE2EDuration="1.576412362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:02:20.574939481 +0000 UTC m=+1725.727940208" watchObservedRunningTime="2026-07-24 15:02:20.576412362 +0000 UTC m=+1725.729413087"
Jul 24 15:02:20.606071 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:20.606026    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8" podStartSLOduration=1.606010492 podStartE2EDuration="1.606010492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:02:20.603972348 +0000 UTC m=+1725.756973074" watchObservedRunningTime="2026-07-24 15:02:20.606010492 +0000 UTC m=+1725.759011220"
Jul 24 15:02:21.350052 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:21.350010    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c508f91-622a-4550-8a49-8c24cba87b1e" path="/var/lib/kubelet/pods/3c508f91-622a-4550-8a49-8c24cba87b1e/volumes"
Jul 24 15:02:21.559019 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:21.558988    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt" event={"ID":"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347","Type":"ContainerStarted","Data":"37eae44be48bf55b73f6d979937b48edf5fcb3b7af58fc4c11f02fc505301cd2"}
Jul 24 15:02:21.559447 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:21.559428    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:21.602895 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:21.602817    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt" podStartSLOduration=3.850039764 podStartE2EDuration="4.602804611s" totalImagesPullingTime="752.764847ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:02:21.601233587 +0000 UTC m=+1726.754234312" watchObservedRunningTime="2026-07-24 15:02:21.602804611 +0000 UTC m=+1726.755805400"
Jul 24 15:02:25.809730 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:25.809701    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:25.896725 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:25.896691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:28.574028 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:28.573998    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:31.560991 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:31.560962    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:31.561373 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:31.561031    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:53.456445 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.456408    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"]
Jul 24 15:02:53.457005 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.456634    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt" podUID="15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" containerName="maas-api" containerID="cri-o://37eae44be48bf55b73f6d979937b48edf5fcb3b7af58fc4c11f02fc505301cd2" gracePeriod=30
Jul 24 15:02:53.519378 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.519350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:53.569613 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.569581    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt" podUID="15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.55:8443/health\": dial tcp 10.132.0.55:8443: connect: connection refused"
Jul 24 15:02:53.572701 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.572639    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"]
Jul 24 15:02:53.572948 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.572892    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r" podUID="f1c9ef36-03c8-4801-8c79-d0c4486a4537" containerName="payload-processing" containerID="cri-o://7ab26844a4a8ef0ed1cd4f1bf22c8199326f766db5f6f7a523e4aada335e14dd" gracePeriod=30
Jul 24 15:02:53.580031 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.579861    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"]
Jul 24 15:02:53.580243 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.580056    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8" podUID="b37d2611-eeae-4f8a-a02e-858957a180fd" containerName="payload-pre-processing" containerID="cri-o://ce02ed9f65f1d46f255e2830d9f8b347be6d5f0c476218e53acf5dc6ce043843" gracePeriod=30
Jul 24 15:02:53.638477 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.638295    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:53.711795 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.711761    2583 generic.go:356] "Generic (PLEG): container finished" podID="15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" containerID="37eae44be48bf55b73f6d979937b48edf5fcb3b7af58fc4c11f02fc505301cd2" exitCode=0
Jul 24 15:02:53.711910 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.711853    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt" event={"ID":"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347","Type":"ContainerDied","Data":"37eae44be48bf55b73f6d979937b48edf5fcb3b7af58fc4c11f02fc505301cd2"}
Jul 24 15:02:53.711910 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.711906    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt" event={"ID":"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347","Type":"ContainerDied","Data":"54eafd1af910d1538dfba6c9fd292e2a6c98a6ff9aa988c1fde8313c2c97a823"}
Jul 24 15:02:53.712036 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.711930    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54eafd1af910d1538dfba6c9fd292e2a6c98a6ff9aa988c1fde8313c2c97a823"
Jul 24 15:02:53.714141 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.714119    2583 generic.go:356] "Generic (PLEG): container finished" podID="b37d2611-eeae-4f8a-a02e-858957a180fd" containerID="ce02ed9f65f1d46f255e2830d9f8b347be6d5f0c476218e53acf5dc6ce043843" exitCode=0
Jul 24 15:02:53.714248 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.714200    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8" event={"ID":"b37d2611-eeae-4f8a-a02e-858957a180fd","Type":"ContainerDied","Data":"ce02ed9f65f1d46f255e2830d9f8b347be6d5f0c476218e53acf5dc6ce043843"}
Jul 24 15:02:53.716104 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.716072    2583 generic.go:356] "Generic (PLEG): container finished" podID="f1c9ef36-03c8-4801-8c79-d0c4486a4537" containerID="7ab26844a4a8ef0ed1cd4f1bf22c8199326f766db5f6f7a523e4aada335e14dd" exitCode=0
Jul 24 15:02:53.716220 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.716107    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r" event={"ID":"f1c9ef36-03c8-4801-8c79-d0c4486a4537","Type":"ContainerDied","Data":"7ab26844a4a8ef0ed1cd4f1bf22c8199326f766db5f6f7a523e4aada335e14dd"}
Jul 24 15:02:53.726570 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.726548    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:53.745217 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.745188    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:53.785125 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.785102    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2wv6\" (UniqueName: \"kubernetes.io/projected/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-kube-api-access-n2wv6\") pod \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") "
Jul 24 15:02:53.785261 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.785162    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-openshift-service-ca\") pod \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") "
Jul 24 15:02:53.785261 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.785191    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls\") pod \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\" (UID: \"15cdd6ea-aef7-4e67-815e-c4f4d9ea2347\") "
Jul 24 15:02:53.785587 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.785561    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" (UID: "15cdd6ea-aef7-4e67-815e-c4f4d9ea2347"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:53.787679 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.787655    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-kube-api-access-n2wv6" (OuterVolumeSpecName: "kube-api-access-n2wv6") pod "15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" (UID: "15cdd6ea-aef7-4e67-815e-c4f4d9ea2347"). InnerVolumeSpecName "kube-api-access-n2wv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:53.787807 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.787694    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" (UID: "15cdd6ea-aef7-4e67-815e-c4f4d9ea2347"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:02:53.826223 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.826201    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:53.842756 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.842735    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:53.886629 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.886600    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c9ef36-03c8-4801-8c79-d0c4486a4537-plugins-config-volume\") pod \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\" (UID: \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\") "
Jul 24 15:02:53.886752 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.886680    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ht9l\" (UniqueName: \"kubernetes.io/projected/f1c9ef36-03c8-4801-8c79-d0c4486a4537-kube-api-access-5ht9l\") pod \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\" (UID: \"f1c9ef36-03c8-4801-8c79-d0c4486a4537\") "
Jul 24 15:02:53.886752 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.886713    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b37d2611-eeae-4f8a-a02e-858957a180fd-plugins-config-volume\") pod \"b37d2611-eeae-4f8a-a02e-858957a180fd\" (UID: \"b37d2611-eeae-4f8a-a02e-858957a180fd\") "
Jul 24 15:02:53.886752 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.886736    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgn56\" (UniqueName: \"kubernetes.io/projected/b37d2611-eeae-4f8a-a02e-858957a180fd-kube-api-access-tgn56\") pod \"b37d2611-eeae-4f8a-a02e-858957a180fd\" (UID: \"b37d2611-eeae-4f8a-a02e-858957a180fd\") "
Jul 24 15:02:53.887013 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.886972    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1c9ef36-03c8-4801-8c79-d0c4486a4537-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1c9ef36-03c8-4801-8c79-d0c4486a4537" (UID: "f1c9ef36-03c8-4801-8c79-d0c4486a4537"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:53.887136 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.887030    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2wv6\" (UniqueName: \"kubernetes.io/projected/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-kube-api-access-n2wv6\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:53.887136 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.887051    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-openshift-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:53.887136 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.887052    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b37d2611-eeae-4f8a-a02e-858957a180fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b37d2611-eeae-4f8a-a02e-858957a180fd" (UID: "b37d2611-eeae-4f8a-a02e-858957a180fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:53.887136 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.887065    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347-maas-api-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:53.888801 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.888778    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b37d2611-eeae-4f8a-a02e-858957a180fd-kube-api-access-tgn56" (OuterVolumeSpecName: "kube-api-access-tgn56") pod "b37d2611-eeae-4f8a-a02e-858957a180fd" (UID: "b37d2611-eeae-4f8a-a02e-858957a180fd"). InnerVolumeSpecName "kube-api-access-tgn56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:53.888896 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.888817    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1c9ef36-03c8-4801-8c79-d0c4486a4537-kube-api-access-5ht9l" (OuterVolumeSpecName: "kube-api-access-5ht9l") pod "f1c9ef36-03c8-4801-8c79-d0c4486a4537" (UID: "f1c9ef36-03c8-4801-8c79-d0c4486a4537"). InnerVolumeSpecName "kube-api-access-5ht9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:53.988334 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.988281    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ht9l\" (UniqueName: \"kubernetes.io/projected/f1c9ef36-03c8-4801-8c79-d0c4486a4537-kube-api-access-5ht9l\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:53.988334 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.988302    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b37d2611-eeae-4f8a-a02e-858957a180fd-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:53.988334 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.988311    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgn56\" (UniqueName: \"kubernetes.io/projected/b37d2611-eeae-4f8a-a02e-858957a180fd-kube-api-access-tgn56\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:53.988334 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:53.988320    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c9ef36-03c8-4801-8c79-d0c4486a4537-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:02:54.722585 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.722528    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"
Jul 24 15:02:54.722936 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.722528    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r" event={"ID":"f1c9ef36-03c8-4801-8c79-d0c4486a4537","Type":"ContainerDied","Data":"b801a8a1f4e05c8fec6300ef3fed10071ace4338aed68c6a9c54847b6d86b5d7"}
Jul 24 15:02:54.722936 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.722662    2583 scope.go:117] "RemoveContainer" containerID="7ab26844a4a8ef0ed1cd4f1bf22c8199326f766db5f6f7a523e4aada335e14dd"
Jul 24 15:02:54.723942 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.723918    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"
Jul 24 15:02:54.724010 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.723958    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8" event={"ID":"b37d2611-eeae-4f8a-a02e-858957a180fd","Type":"ContainerDied","Data":"d0d47d8dfc90fc19cd4afa01a3aedf808fa0244580b8919785858d0903b70ca5"}
Jul 24 15:02:54.724135 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.724118    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"
Jul 24 15:02:54.732395 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.732378    2583 scope.go:117] "RemoveContainer" containerID="ce02ed9f65f1d46f255e2830d9f8b347be6d5f0c476218e53acf5dc6ce043843"
Jul 24 15:02:54.767743 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.767722    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"]
Jul 24 15:02:54.775005 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.774986    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r"]
Jul 24 15:02:54.816728 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.816709    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"]
Jul 24 15:02:54.819370 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.819351    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8"]
Jul 24 15:02:54.843689 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.843663    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"]
Jul 24 15:02:54.850693 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:54.850674    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-47d0d404-5c85bdd959-2vrgt"]
Jul 24 15:02:55.342883 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:55.342855    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" path="/var/lib/kubelet/pods/15cdd6ea-aef7-4e67-815e-c4f4d9ea2347/volumes"
Jul 24 15:02:55.343230 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:55.343213    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b37d2611-eeae-4f8a-a02e-858957a180fd" path="/var/lib/kubelet/pods/b37d2611-eeae-4f8a-a02e-858957a180fd/volumes"
Jul 24 15:02:55.343578 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:55.343562    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1c9ef36-03c8-4801-8c79-d0c4486a4537" path="/var/lib/kubelet/pods/f1c9ef36-03c8-4801-8c79-d0c4486a4537/volumes"
Jul 24 15:02:58.035321 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:58.035280    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:58.274403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:58.274356    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:58.531486 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:58.531429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:58.784698 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:58.784621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:59.249535 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249501    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"]
Jul 24 15:02:59.249861 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249836    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" containerName="maas-api"
Jul 24 15:02:59.249861 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249847    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" containerName="maas-api"
Jul 24 15:02:59.249861 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249856    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b37d2611-eeae-4f8a-a02e-858957a180fd" containerName="payload-pre-processing"
Jul 24 15:02:59.249861 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249861    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b37d2611-eeae-4f8a-a02e-858957a180fd" containerName="payload-pre-processing"
Jul 24 15:02:59.249999 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249893    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1c9ef36-03c8-4801-8c79-d0c4486a4537" containerName="payload-processing"
Jul 24 15:02:59.249999 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249899    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1c9ef36-03c8-4801-8c79-d0c4486a4537" containerName="payload-processing"
Jul 24 15:02:59.249999 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249950    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="15cdd6ea-aef7-4e67-815e-c4f4d9ea2347" containerName="maas-api"
Jul 24 15:02:59.249999 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249960    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1c9ef36-03c8-4801-8c79-d0c4486a4537" containerName="payload-processing"
Jul 24 15:02:59.249999 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.249966    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b37d2611-eeae-4f8a-a02e-858957a180fd" containerName="payload-pre-processing"
Jul 24 15:02:59.254481 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.254443    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.258003 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.257983    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a1550b4d-openshift-default-dockercfg-rj7h2\""
Jul 24 15:02:59.276279 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.276256    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"]
Jul 24 15:02:59.316741 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.316707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:59.325866 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.325844    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-podinfo\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.325948 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.325880    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-token\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.325948 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.325908    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-credential-socket\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.326040 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.325953    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-envoy\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.326040 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.325974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dck2j\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-kube-api-access-dck2j\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.326040 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.325998    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-socket\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.326040 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.326020    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-certs\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.326180 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.326044    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-data\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.326180 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.326061    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istiod-ca-cert\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.426707 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.426677    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-token\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.426846 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.426713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-credential-socket\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.426846 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.426731    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-envoy\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.426846 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.426753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dck2j\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-kube-api-access-dck2j\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.426846 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.426775    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-socket\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.427058 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.426938    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-certs\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.427058 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.426986    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-data\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.427058 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.427010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istiod-ca-cert\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.427253 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.427062    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-podinfo\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.427253 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.427159    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-credential-socket\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.427253 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.427162    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-socket\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.427383 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.427361    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-data\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.427428 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.427410    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-certs\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.427654 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.427639    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istiod-ca-cert\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.428959 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.428941    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-envoy\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.429250 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.429234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-podinfo\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.441049 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.441028    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dck2j\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-kube-api-access-dck2j\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.448763 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.448738    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-token\") pod \"e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") " pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.478343 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.478305    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:59.564126 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.564071    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:02:59.743515 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.743487    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"]
Jul 24 15:02:59.744600 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:02:59.744568    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4dabb32_4fd3_4896_8c5c_6bd74aea48c8.slice/crio-33dadfe7360e7a9dd2156dfafc257017c16f4b2d3f5a371dc7b64fa1a73474e9 WatchSource:0}: Error finding container 33dadfe7360e7a9dd2156dfafc257017c16f4b2d3f5a371dc7b64fa1a73474e9: Status 404 returned error can't find the container with id 33dadfe7360e7a9dd2156dfafc257017c16f4b2d3f5a371dc7b64fa1a73474e9
Jul 24 15:02:59.746671 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.746642    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:02:59.746747 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.746702    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:02:59.746747 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:02:59.746729    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:03:00.755448 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:00.755416    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" event={"ID":"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8","Type":"ContainerStarted","Data":"f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798"}
Jul 24 15:03:00.755834 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:00.755482    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" event={"ID":"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8","Type":"ContainerStarted","Data":"33dadfe7360e7a9dd2156dfafc257017c16f4b2d3f5a371dc7b64fa1a73474e9"}
Jul 24 15:03:00.786884 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:00.786841    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" podStartSLOduration=1.7868252 podStartE2EDuration="1.7868252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:02:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:00.785969494 +0000 UTC m=+1765.938970222" watchObservedRunningTime="2026-07-24 15:03:00.7868252 +0000 UTC m=+1765.939825925"
Jul 24 15:03:01.564836 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:01.564804    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:01.569547 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:01.569524    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:01.760124 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:01.760096    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:01.760908 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:01.760889    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:02.253025 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.252980    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:02.388847 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.388812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:02.401203 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.401175    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"]
Jul 24 15:03:02.404621 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.404605    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.407389 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.407366    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a1550b4d-serving-cert\""
Jul 24 15:03:02.431372 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.431320    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"]
Jul 24 15:03:02.452773 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.452751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf011fd7-4685-4535-8873-30ed787c2701-openshift-service-ca\") pod \"maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.452937 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.452883    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf011fd7-4685-4535-8873-30ed787c2701-maas-api-tls\") pod \"maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.453010 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.452963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44mf7\" (UniqueName: \"kubernetes.io/projected/cf011fd7-4685-4535-8873-30ed787c2701-kube-api-access-44mf7\") pod \"maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.554073 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.553988    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf011fd7-4685-4535-8873-30ed787c2701-maas-api-tls\") pod \"maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.554073 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.554040    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44mf7\" (UniqueName: \"kubernetes.io/projected/cf011fd7-4685-4535-8873-30ed787c2701-kube-api-access-44mf7\") pod \"maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.554278 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.554079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf011fd7-4685-4535-8873-30ed787c2701-openshift-service-ca\") pod \"maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.554765 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.554741    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf011fd7-4685-4535-8873-30ed787c2701-openshift-service-ca\") pod \"maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.556433 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.556410    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf011fd7-4685-4535-8873-30ed787c2701-maas-api-tls\") pod \"maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.572989 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.572958    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44mf7\" (UniqueName: \"kubernetes.io/projected/cf011fd7-4685-4535-8873-30ed787c2701-kube-api-access-44mf7\") pod \"maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.583403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.583373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:02.715063 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.715042    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:02.968826 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:02.968797    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:03.079849 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:03.079819    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"]
Jul 24 15:03:03.082723 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:03:03.082700    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf011fd7_4685_4535_8873_30ed787c2701.slice/crio-4bb304fda562e2437649b9c142b05feb0e8c726041438d16ebff1c68d2d8d00c WatchSource:0}: Error finding container 4bb304fda562e2437649b9c142b05feb0e8c726041438d16ebff1c68d2d8d00c: Status 404 returned error can't find the container with id 4bb304fda562e2437649b9c142b05feb0e8c726041438d16ebff1c68d2d8d00c
Jul 24 15:03:03.773952 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:03.773922    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx" event={"ID":"cf011fd7-4685-4535-8873-30ed787c2701","Type":"ContainerStarted","Data":"0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc"}
Jul 24 15:03:03.774106 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:03.773982    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx" event={"ID":"cf011fd7-4685-4535-8873-30ed787c2701","Type":"ContainerStarted","Data":"4bb304fda562e2437649b9c142b05feb0e8c726041438d16ebff1c68d2d8d00c"}
Jul 24 15:03:03.774184 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:03.774165    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:03.830347 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:03.830308    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx" podStartSLOduration=1.248571895 podStartE2EDuration="1.830295754s" totalImagesPullingTime="581.723859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:03.828668132 +0000 UTC m=+1768.981668857" watchObservedRunningTime="2026-07-24 15:03:03.830295754 +0000 UTC m=+1768.983296479"
Jul 24 15:03:03.898208 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:03.898177    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:04.106907 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.106875    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:04.241142 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.241105    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"]
Jul 24 15:03:04.244685 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.244660    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:04.248129 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.248107    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a1550b4d\""
Jul 24 15:03:04.248250 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.248109    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a1550b4d-dockercfg-zb2dr\""
Jul 24 15:03:04.264944 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.264922    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"]
Jul 24 15:03:04.370167 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.370105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89slb\" (UniqueName: \"kubernetes.io/projected/cf7331a9-e083-492a-9574-60a0743767b9-kube-api-access-89slb\") pod \"payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4\" (UID: \"cf7331a9-e083-492a-9574-60a0743767b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:04.370273 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.370179    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7331a9-e083-492a-9574-60a0743767b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4\" (UID: \"cf7331a9-e083-492a-9574-60a0743767b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:04.375521 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.375495    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"]
Jul 24 15:03:04.379029 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.379010    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:04.405745 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.405714    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"]
Jul 24 15:03:04.470962 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.470938    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-plugins-config-volume\") pod \"payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x\" (UID: \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\") " pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:04.471069 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.470972    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7331a9-e083-492a-9574-60a0743767b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4\" (UID: \"cf7331a9-e083-492a-9574-60a0743767b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:04.471069 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.471055    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89slb\" (UniqueName: \"kubernetes.io/projected/cf7331a9-e083-492a-9574-60a0743767b9-kube-api-access-89slb\") pod \"payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4\" (UID: \"cf7331a9-e083-492a-9574-60a0743767b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:04.471164 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.471112    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bllkj\" (UniqueName: \"kubernetes.io/projected/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-kube-api-access-bllkj\") pod \"payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x\" (UID: \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\") " pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:04.471738 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.471720    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7331a9-e083-492a-9574-60a0743767b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4\" (UID: \"cf7331a9-e083-492a-9574-60a0743767b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:04.479987 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.479968    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89slb\" (UniqueName: \"kubernetes.io/projected/cf7331a9-e083-492a-9574-60a0743767b9-kube-api-access-89slb\") pod \"payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4\" (UID: \"cf7331a9-e083-492a-9574-60a0743767b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:04.533533 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.533510    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:04.555543 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.555518    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:04.572003 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.571851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-plugins-config-volume\") pod \"payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x\" (UID: \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\") " pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:04.572003 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.571928    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bllkj\" (UniqueName: \"kubernetes.io/projected/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-kube-api-access-bllkj\") pod \"payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x\" (UID: \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\") " pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:04.572714 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.572684    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-plugins-config-volume\") pod \"payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x\" (UID: \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\") " pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:04.585564 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.585544    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bllkj\" (UniqueName: \"kubernetes.io/projected/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-kube-api-access-bllkj\") pod \"payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x\" (UID: \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\") " pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:04.690702 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.690671    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:04.851167 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:03:04.851133    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e9c1a0_41d6_4480_a9ec_a1c26da3801f.slice/crio-96c25fc55efdb036b27b3017952b978c3c027d2c3e41779099d5648ff99d71c8 WatchSource:0}: Error finding container 96c25fc55efdb036b27b3017952b978c3c027d2c3e41779099d5648ff99d71c8: Status 404 returned error can't find the container with id 96c25fc55efdb036b27b3017952b978c3c027d2c3e41779099d5648ff99d71c8
Jul 24 15:03:04.852493 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.852464    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"]
Jul 24 15:03:04.922571 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:04.922546    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"]
Jul 24 15:03:04.925253 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:03:04.925225    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7331a9_e083_492a_9574_60a0743767b9.slice/crio-a0877301491d6515203f9936ea4c2b51273070ba55a870dd4e1dfaffe013cd56 WatchSource:0}: Error finding container a0877301491d6515203f9936ea4c2b51273070ba55a870dd4e1dfaffe013cd56: Status 404 returned error can't find the container with id a0877301491d6515203f9936ea4c2b51273070ba55a870dd4e1dfaffe013cd56
Jul 24 15:03:05.785246 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:05.785210    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x" event={"ID":"49e9c1a0-41d6-4480-a9ec-a1c26da3801f","Type":"ContainerStarted","Data":"f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352"}
Jul 24 15:03:05.785677 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:05.785272    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x" event={"ID":"49e9c1a0-41d6-4480-a9ec-a1c26da3801f","Type":"ContainerStarted","Data":"96c25fc55efdb036b27b3017952b978c3c027d2c3e41779099d5648ff99d71c8"}
Jul 24 15:03:05.785677 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:05.785301    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:05.786709 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:05.786687    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4" event={"ID":"cf7331a9-e083-492a-9574-60a0743767b9","Type":"ContainerStarted","Data":"6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9"}
Jul 24 15:03:05.786799 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:05.786725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4" event={"ID":"cf7331a9-e083-492a-9574-60a0743767b9","Type":"ContainerStarted","Data":"a0877301491d6515203f9936ea4c2b51273070ba55a870dd4e1dfaffe013cd56"}
Jul 24 15:03:05.786837 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:05.786807    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:05.851667 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:05.851625    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x" podStartSLOduration=1.8516106639999999 podStartE2EDuration="1.851610664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:05.851112696 +0000 UTC m=+1771.004113419" watchObservedRunningTime="2026-07-24 15:03:05.851610664 +0000 UTC m=+1771.004611391"
Jul 24 15:03:05.883723 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:05.883689    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4" podStartSLOduration=1.883676753 podStartE2EDuration="1.883676753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:05.879613582 +0000 UTC m=+1771.032614308" watchObservedRunningTime="2026-07-24 15:03:05.883676753 +0000 UTC m=+1771.036677478"
Jul 24 15:03:09.784304 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:09.784273    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:16.793519 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:16.793472    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:16.793916 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:16.793597    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:17.213589 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:17.212673    2583 patch_prober.go:28] interesting pod/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:34390->10.132.0.58:15021: read: connection reset by peer" start-of-body=
Jul 24 15:03:17.213589 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:17.212732    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:34390->10.132.0.58:15021: read: connection reset by peer"
Jul 24 15:03:17.844893 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:17.844863    2583 generic.go:356] "Generic (PLEG): container finished" podID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerID="f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798" exitCode=0
Jul 24 15:03:17.845228 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:17.844934    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" event={"ID":"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8","Type":"ContainerDied","Data":"f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798"}
Jul 24 15:03:17.845299 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:17.845254    2583 scope.go:117] "RemoveContainer" containerID="f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798"
Jul 24 15:03:17.846182 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:17.846159    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:03:17.846252 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:17.846222    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:03:17.846291 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:17.846250    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:03:18.851513 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:18.851477    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" event={"ID":"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8","Type":"ContainerStarted","Data":"1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f"}
Jul 24 15:03:19.564986 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:19.564950    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:19.565211 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:19.565184    2583 patch_prober.go:28] interesting pod/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 24 15:03:19.565273 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:19.565251    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 24 15:03:20.565379 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:20.565342    2583 patch_prober.go:28] interesting pod/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 24 15:03:20.565760 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:20.565406    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 24 15:03:21.565043 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:21.565008    2583 patch_prober.go:28] interesting pod/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 24 15:03:21.565204 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:21.565065    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 24 15:03:22.564964 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:22.564931    2583 patch_prober.go:28] interesting pod/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 24 15:03:22.565301 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:22.565001    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 24 15:03:23.567549 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:23.567519    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:23.567914 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:23.567779    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:23.568504 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:23.568487    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:38.848945 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:38.848906    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"]
Jul 24 15:03:38.849520 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:38.849187    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx" podUID="cf011fd7-4685-4535-8873-30ed787c2701" containerName="maas-api" containerID="cri-o://0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc" gracePeriod=30
Jul 24 15:03:38.928750 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:38.928724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:38.994154 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:38.994123    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"]
Jul 24 15:03:38.994345 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:38.994322    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x" podUID="49e9c1a0-41d6-4480-a9ec-a1c26da3801f" containerName="payload-processing" containerID="cri-o://f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352" gracePeriod=30
Jul 24 15:03:39.022372 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.022345    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"]
Jul 24 15:03:39.022596 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.022570    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4" podUID="cf7331a9-e083-492a-9574-60a0743767b9" containerName="payload-pre-processing" containerID="cri-o://6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9" gracePeriod=30
Jul 24 15:03:39.148029 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.148008    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:39.197279 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.197251    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:39.249055 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.249019    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf011fd7-4685-4535-8873-30ed787c2701-openshift-service-ca\") pod \"cf011fd7-4685-4535-8873-30ed787c2701\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") "
Jul 24 15:03:39.249236 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.249203    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf011fd7-4685-4535-8873-30ed787c2701-maas-api-tls\") pod \"cf011fd7-4685-4535-8873-30ed787c2701\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") "
Jul 24 15:03:39.249307 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.249255    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44mf7\" (UniqueName: \"kubernetes.io/projected/cf011fd7-4685-4535-8873-30ed787c2701-kube-api-access-44mf7\") pod \"cf011fd7-4685-4535-8873-30ed787c2701\" (UID: \"cf011fd7-4685-4535-8873-30ed787c2701\") "
Jul 24 15:03:39.250219 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.249945    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf011fd7-4685-4535-8873-30ed787c2701-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf011fd7-4685-4535-8873-30ed787c2701" (UID: "cf011fd7-4685-4535-8873-30ed787c2701"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:03:39.250566 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.250531    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf011fd7-4685-4535-8873-30ed787c2701-openshift-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:39.253854 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.253806    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf011fd7-4685-4535-8873-30ed787c2701-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf011fd7-4685-4535-8873-30ed787c2701" (UID: "cf011fd7-4685-4535-8873-30ed787c2701"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:03:39.256767 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.256734    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf011fd7-4685-4535-8873-30ed787c2701-kube-api-access-44mf7" (OuterVolumeSpecName: "kube-api-access-44mf7") pod "cf011fd7-4685-4535-8873-30ed787c2701" (UID: "cf011fd7-4685-4535-8873-30ed787c2701"). InnerVolumeSpecName "kube-api-access-44mf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:03:39.299627 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.299597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:39.303633 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.303613    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:39.309175 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.309157    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:39.351480 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.351435    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf011fd7-4685-4535-8873-30ed787c2701-maas-api-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:39.351600 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.351486    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44mf7\" (UniqueName: \"kubernetes.io/projected/cf011fd7-4685-4535-8873-30ed787c2701-kube-api-access-44mf7\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:39.452582 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.452531    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bllkj\" (UniqueName: \"kubernetes.io/projected/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-kube-api-access-bllkj\") pod \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\" (UID: \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\") "
Jul 24 15:03:39.452582 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.452563    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7331a9-e083-492a-9574-60a0743767b9-plugins-config-volume\") pod \"cf7331a9-e083-492a-9574-60a0743767b9\" (UID: \"cf7331a9-e083-492a-9574-60a0743767b9\") "
Jul 24 15:03:39.452706 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.452588    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89slb\" (UniqueName: \"kubernetes.io/projected/cf7331a9-e083-492a-9574-60a0743767b9-kube-api-access-89slb\") pod \"cf7331a9-e083-492a-9574-60a0743767b9\" (UID: \"cf7331a9-e083-492a-9574-60a0743767b9\") "
Jul 24 15:03:39.452825 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.452805    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-plugins-config-volume\") pod \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\" (UID: \"49e9c1a0-41d6-4480-a9ec-a1c26da3801f\") "
Jul 24 15:03:39.452964 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.452942    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf7331a9-e083-492a-9574-60a0743767b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf7331a9-e083-492a-9574-60a0743767b9" (UID: "cf7331a9-e083-492a-9574-60a0743767b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:03:39.453103 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.453081    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7331a9-e083-492a-9574-60a0743767b9-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:39.453216 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.453123    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49e9c1a0-41d6-4480-a9ec-a1c26da3801f" (UID: "49e9c1a0-41d6-4480-a9ec-a1c26da3801f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:03:39.454548 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.454528    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf7331a9-e083-492a-9574-60a0743767b9-kube-api-access-89slb" (OuterVolumeSpecName: "kube-api-access-89slb") pod "cf7331a9-e083-492a-9574-60a0743767b9" (UID: "cf7331a9-e083-492a-9574-60a0743767b9"). InnerVolumeSpecName "kube-api-access-89slb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:03:39.454916 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.454899    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-kube-api-access-bllkj" (OuterVolumeSpecName: "kube-api-access-bllkj") pod "49e9c1a0-41d6-4480-a9ec-a1c26da3801f" (UID: "49e9c1a0-41d6-4480-a9ec-a1c26da3801f"). InnerVolumeSpecName "kube-api-access-bllkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:03:39.553975 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.553952    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89slb\" (UniqueName: \"kubernetes.io/projected/cf7331a9-e083-492a-9574-60a0743767b9-kube-api-access-89slb\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:39.553975 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.553974    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:39.554087 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.553983    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bllkj\" (UniqueName: \"kubernetes.io/projected/49e9c1a0-41d6-4480-a9ec-a1c26da3801f-kube-api-access-bllkj\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:39.953672 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.953643    2583 generic.go:356] "Generic (PLEG): container finished" podID="cf7331a9-e083-492a-9574-60a0743767b9" containerID="6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9" exitCode=0
Jul 24 15:03:39.954030 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.953691    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"
Jul 24 15:03:39.954030 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.953709    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4" event={"ID":"cf7331a9-e083-492a-9574-60a0743767b9","Type":"ContainerDied","Data":"6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9"}
Jul 24 15:03:39.954030 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.953765    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4" event={"ID":"cf7331a9-e083-492a-9574-60a0743767b9","Type":"ContainerDied","Data":"a0877301491d6515203f9936ea4c2b51273070ba55a870dd4e1dfaffe013cd56"}
Jul 24 15:03:39.954030 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.953795    2583 scope.go:117] "RemoveContainer" containerID="6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9"
Jul 24 15:03:39.955216 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.955197    2583 generic.go:356] "Generic (PLEG): container finished" podID="cf011fd7-4685-4535-8873-30ed787c2701" containerID="0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc" exitCode=0
Jul 24 15:03:39.955328 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.955241    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"
Jul 24 15:03:39.955389 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.955347    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx" event={"ID":"cf011fd7-4685-4535-8873-30ed787c2701","Type":"ContainerDied","Data":"0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc"}
Jul 24 15:03:39.955389 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.955385    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx" event={"ID":"cf011fd7-4685-4535-8873-30ed787c2701","Type":"ContainerDied","Data":"4bb304fda562e2437649b9c142b05feb0e8c726041438d16ebff1c68d2d8d00c"}
Jul 24 15:03:39.957007 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.956983    2583 generic.go:356] "Generic (PLEG): container finished" podID="49e9c1a0-41d6-4480-a9ec-a1c26da3801f" containerID="f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352" exitCode=0
Jul 24 15:03:39.957118 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.957010    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x" event={"ID":"49e9c1a0-41d6-4480-a9ec-a1c26da3801f","Type":"ContainerDied","Data":"f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352"}
Jul 24 15:03:39.957118 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.957044    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"
Jul 24 15:03:39.957235 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.957052    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x" event={"ID":"49e9c1a0-41d6-4480-a9ec-a1c26da3801f","Type":"ContainerDied","Data":"96c25fc55efdb036b27b3017952b978c3c027d2c3e41779099d5648ff99d71c8"}
Jul 24 15:03:39.962993 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.962929    2583 scope.go:117] "RemoveContainer" containerID="6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9"
Jul 24 15:03:39.963285 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:03:39.963266    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9\": container with ID starting with 6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9 not found: ID does not exist" containerID="6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9"
Jul 24 15:03:39.963374 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.963294    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9"} err="failed to get container status \"6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9\": rpc error: code = NotFound desc = could not find container \"6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9\": container with ID starting with 6cf4ec0b2e1ffe753da40a576cc1a55dd451fa93587fd75544dc9daec52f6ab9 not found: ID does not exist"
Jul 24 15:03:39.963374 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.963317    2583 scope.go:117] "RemoveContainer" containerID="0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc"
Jul 24 15:03:39.970946 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.970929    2583 scope.go:117] "RemoveContainer" containerID="0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc"
Jul 24 15:03:39.971196 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:03:39.971179    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc\": container with ID starting with 0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc not found: ID does not exist" containerID="0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc"
Jul 24 15:03:39.971233 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.971202    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc"} err="failed to get container status \"0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc\": rpc error: code = NotFound desc = could not find container \"0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc\": container with ID starting with 0cef1d17a107338a4daf3b08dd83f42d5c4d2844eaf3acce5f6cd8e3e19f2efc not found: ID does not exist"
Jul 24 15:03:39.971233 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.971216    2583 scope.go:117] "RemoveContainer" containerID="f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352"
Jul 24 15:03:39.978824 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.978806    2583 scope.go:117] "RemoveContainer" containerID="f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352"
Jul 24 15:03:39.979068 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:03:39.979050    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352\": container with ID starting with f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352 not found: ID does not exist" containerID="f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352"
Jul 24 15:03:39.979121 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.979078    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352"} err="failed to get container status \"f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352\": rpc error: code = NotFound desc = could not find container \"f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352\": container with ID starting with f04a7d52e02326d4f5664dd0e2a2fd9af4bb400f54bbf806f8ab625220223352 not found: ID does not exist"
Jul 24 15:03:39.979288 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.979273    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"]
Jul 24 15:03:39.982480 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.982446    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1550b4d-6966b699d9-n4zcx"]
Jul 24 15:03:39.994928 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:39.994909    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"]
Jul 24 15:03:40.001182 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:40.001162    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x"]
Jul 24 15:03:40.011725 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:40.011706    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"]
Jul 24 15:03:40.019186 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:40.019166    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4"]
Jul 24 15:03:41.343281 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:41.343252    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e9c1a0-41d6-4480-a9ec-a1c26da3801f" path="/var/lib/kubelet/pods/49e9c1a0-41d6-4480-a9ec-a1c26da3801f/volumes"
Jul 24 15:03:41.343675 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:41.343660    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf011fd7-4685-4535-8873-30ed787c2701" path="/var/lib/kubelet/pods/cf011fd7-4685-4535-8873-30ed787c2701/volumes"
Jul 24 15:03:41.343971 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:41.343960    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf7331a9-e083-492a-9574-60a0743767b9" path="/var/lib/kubelet/pods/cf7331a9-e083-492a-9574-60a0743767b9/volumes"
Jul 24 15:03:43.257176 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:43.257128    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:44.477988 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:44.477952    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"]
Jul 24 15:03:44.478440 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:44.478154    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy" containerID="cri-o://1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f" gracePeriod=30
Jul 24 15:03:44.505646 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:44.505610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:44.748876 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:44.748802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:45.007657 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:45.007585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:45.319481 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:45.319371    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:45.420400 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:45.420366    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:47.972941 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:47.972912    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:48.068494 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.068442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:48.206490 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206444    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"]
Jul 24 15:03:48.206857 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206844    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7331a9-e083-492a-9574-60a0743767b9" containerName="payload-pre-processing"
Jul 24 15:03:48.206913 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206859    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7331a9-e083-492a-9574-60a0743767b9" containerName="payload-pre-processing"
Jul 24 15:03:48.206913 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206876    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf011fd7-4685-4535-8873-30ed787c2701" containerName="maas-api"
Jul 24 15:03:48.206913 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206882    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf011fd7-4685-4535-8873-30ed787c2701" containerName="maas-api"
Jul 24 15:03:48.206913 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206889    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e9c1a0-41d6-4480-a9ec-a1c26da3801f" containerName="payload-processing"
Jul 24 15:03:48.206913 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206895    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e9c1a0-41d6-4480-a9ec-a1c26da3801f" containerName="payload-processing"
Jul 24 15:03:48.207063 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206942    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf7331a9-e083-492a-9574-60a0743767b9" containerName="payload-pre-processing"
Jul 24 15:03:48.207063 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206952    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf011fd7-4685-4535-8873-30ed787c2701" containerName="maas-api"
Jul 24 15:03:48.207063 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.206958    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e9c1a0-41d6-4480-a9ec-a1c26da3801f" containerName="payload-processing"
Jul 24 15:03:48.211781 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.211759    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.214325 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.214304    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-84b622a4-serving-cert\""
Jul 24 15:03:48.219057 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.219033    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"]
Jul 24 15:03:48.244271 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.244215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:48.317545 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.317510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57fsl\" (UniqueName: \"kubernetes.io/projected/3c14e193-2b4b-478c-824c-fc1208f65c32-kube-api-access-57fsl\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.317706 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.317570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c14e193-2b4b-478c-824c-fc1208f65c32-openshift-service-ca\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.317706 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.317599    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.418989 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.418963    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57fsl\" (UniqueName: \"kubernetes.io/projected/3c14e193-2b4b-478c-824c-fc1208f65c32-kube-api-access-57fsl\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.419116 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.419004    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c14e193-2b4b-478c-824c-fc1208f65c32-openshift-service-ca\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.419116 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.419028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.419196 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:03:48.419157    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-serving-cert: secret "maas-api-e2e-derive-84b622a4-serving-cert" not found
Jul 24 15:03:48.419232 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:03:48.419220    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls podName:3c14e193-2b4b-478c-824c-fc1208f65c32 nodeName:}" failed. No retries permitted until 2026-07-24 15:03:48.919200246 +0000 UTC m=+1814.072200963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls") pod "maas-api-e2e-derive-84b622a4-644dcd745-bkgxs" (UID: "3c14e193-2b4b-478c-824c-fc1208f65c32") : secret "maas-api-e2e-derive-84b622a4-serving-cert" not found
Jul 24 15:03:48.419653 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.419635    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c14e193-2b4b-478c-824c-fc1208f65c32-openshift-service-ca\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.427855 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.427836    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57fsl\" (UniqueName: \"kubernetes.io/projected/3c14e193-2b4b-478c-824c-fc1208f65c32-kube-api-access-57fsl\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.828297 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.828272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:48.924099 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.924066    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:48.924230 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:03:48.924182    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-serving-cert: secret "maas-api-e2e-derive-84b622a4-serving-cert" not found
Jul 24 15:03:48.924272 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:03:48.924243    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls podName:3c14e193-2b4b-478c-824c-fc1208f65c32 nodeName:}" failed. No retries permitted until 2026-07-24 15:03:49.924228635 +0000 UTC m=+1815.077229339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls") pod "maas-api-e2e-derive-84b622a4-644dcd745-bkgxs" (UID: "3c14e193-2b4b-478c-824c-fc1208f65c32") : secret "maas-api-e2e-derive-84b622a4-serving-cert" not found
Jul 24 15:03:48.929281 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.929255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:48.955867 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.955841    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"]
Jul 24 15:03:48.959359 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.959343    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:03:48.962990 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.962968    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-84b622a4\""
Jul 24 15:03:48.976078 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.976059    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"]
Jul 24 15:03:48.994051 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.994028    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"]
Jul 24 15:03:48.997355 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:48.997333    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:03:49.001911 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.001895    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-84b622a4-dockercfg-thzxt\""
Jul 24 15:03:49.013033 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.013010    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"]
Jul 24 15:03:49.126347 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.126301    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjn4c\" (UniqueName: \"kubernetes.io/projected/abb11d3d-4996-4a2b-a242-86340124c09c-kube-api-access-qjn4c\") pod \"payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc\" (UID: \"abb11d3d-4996-4a2b-a242-86340124c09c\") " pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:03:49.126573 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.126362    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ngd4\" (UniqueName: \"kubernetes.io/projected/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-kube-api-access-2ngd4\") pod \"payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx\" (UID: \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:03:49.126573 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.126445    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx\" (UID: \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:03:49.126573 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.126493    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb11d3d-4996-4a2b-a242-86340124c09c-plugins-config-volume\") pod \"payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc\" (UID: \"abb11d3d-4996-4a2b-a242-86340124c09c\") " pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:03:49.227864 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.227833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ngd4\" (UniqueName: \"kubernetes.io/projected/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-kube-api-access-2ngd4\") pod \"payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx\" (UID: \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:03:49.228021 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.227890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx\" (UID: \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:03:49.228021 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.227910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb11d3d-4996-4a2b-a242-86340124c09c-plugins-config-volume\") pod \"payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc\" (UID: \"abb11d3d-4996-4a2b-a242-86340124c09c\") " pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:03:49.228021 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.227981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjn4c\" (UniqueName: \"kubernetes.io/projected/abb11d3d-4996-4a2b-a242-86340124c09c-kube-api-access-qjn4c\") pod \"payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc\" (UID: \"abb11d3d-4996-4a2b-a242-86340124c09c\") " pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:03:49.228640 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.228615    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx\" (UID: \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:03:49.228740 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.228645    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb11d3d-4996-4a2b-a242-86340124c09c-plugins-config-volume\") pod \"payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc\" (UID: \"abb11d3d-4996-4a2b-a242-86340124c09c\") " pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:03:49.236515 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.236482    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjn4c\" (UniqueName: \"kubernetes.io/projected/abb11d3d-4996-4a2b-a242-86340124c09c-kube-api-access-qjn4c\") pod \"payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc\" (UID: \"abb11d3d-4996-4a2b-a242-86340124c09c\") " pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:03:49.236673 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.236651    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ngd4\" (UniqueName: \"kubernetes.io/projected/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-kube-api-access-2ngd4\") pod \"payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx\" (UID: \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:03:49.270316 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.270293    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:03:49.307683 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.307654    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:03:49.310859 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.310804    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:49.414732 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.414625    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"]
Jul 24 15:03:49.417444 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:03:49.417421    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabb11d3d_4996_4a2b_a242_86340124c09c.slice/crio-eefb27568c372ef57fc175348fac8f995b8f1e7030133ef8d8ca1530aca754b9 WatchSource:0}: Error finding container eefb27568c372ef57fc175348fac8f995b8f1e7030133ef8d8ca1530aca754b9: Status 404 returned error can't find the container with id eefb27568c372ef57fc175348fac8f995b8f1e7030133ef8d8ca1530aca754b9
Jul 24 15:03:49.464820 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.464800    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"]
Jul 24 15:03:49.471793 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:03:49.471767    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d3e63a_fd2b_4082_9f20_14fe3e116bdc.slice/crio-421c11a47909010aaeb3044fda8dd1606519bedbe74c3daa39461af54c17a00d WatchSource:0}: Error finding container 421c11a47909010aaeb3044fda8dd1606519bedbe74c3daa39461af54c17a00d: Status 404 returned error can't find the container with id 421c11a47909010aaeb3044fda8dd1606519bedbe74c3daa39461af54c17a00d
Jul 24 15:03:49.724828 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.724805    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:49.835525 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835499    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-credential-socket\") pod \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") "
Jul 24 15:03:49.835697 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835536    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-certs\") pod \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") "
Jul 24 15:03:49.835697 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835589    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-podinfo\") pod \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") "
Jul 24 15:03:49.835697 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835626    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dck2j\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-kube-api-access-dck2j\") pod \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") "
Jul 24 15:03:49.835697 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835662    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istiod-ca-cert\") pod \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") "
Jul 24 15:03:49.835895 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835725    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-token\") pod \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") "
Jul 24 15:03:49.835895 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835741    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" (UID: "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:49.835895 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835765    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-data\") pod \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") "
Jul 24 15:03:49.835895 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835799    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-socket\") pod \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") "
Jul 24 15:03:49.835895 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835834    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-envoy\") pod \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\" (UID: \"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8\") "
Jul 24 15:03:49.836127 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.835986    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" (UID: "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:03:49.836182 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.836131    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:49.836182 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.836149    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:49.836182 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.836156    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" (UID: "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:49.836355 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.836335    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" (UID: "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:49.837143 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.837115    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-data" (OuterVolumeSpecName: "istio-data") pod "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" (UID: "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:49.838661 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.838628    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-token" (OuterVolumeSpecName: "istio-token") pod "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" (UID: "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:03:49.838952 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.838922    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" (UID: "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:03:49.839298 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.839274    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" (UID: "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:49.839298 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.839283    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-kube-api-access-dck2j" (OuterVolumeSpecName: "kube-api-access-dck2j") pod "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" (UID: "c4dabb32-4fd3-4896-8c5c-6bd74aea48c8"). InnerVolumeSpecName "kube-api-access-dck2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:03:49.936843 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.936817    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:49.936972 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.936922    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:49.936972 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.936937    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:49.936972 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.936951    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:49.936972 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.936964    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:49.937175 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.936975    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:49.937175 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.936987    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:49.937175 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.936998    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dck2j\" (UniqueName: \"kubernetes.io/projected/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8-kube-api-access-dck2j\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:03:49.938859 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:49.938839    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls\") pod \"maas-api-e2e-derive-84b622a4-644dcd745-bkgxs\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:50.015683 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.015629    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx" event={"ID":"06d3e63a-fd2b-4082-9f20-14fe3e116bdc","Type":"ContainerStarted","Data":"062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389"}
Jul 24 15:03:50.015683 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.015675    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx" event={"ID":"06d3e63a-fd2b-4082-9f20-14fe3e116bdc","Type":"ContainerStarted","Data":"421c11a47909010aaeb3044fda8dd1606519bedbe74c3daa39461af54c17a00d"}
Jul 24 15:03:50.016030 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.015722    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:03:50.017274 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.017252    2583 generic.go:356] "Generic (PLEG): container finished" podID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerID="1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f" exitCode=0
Jul 24 15:03:50.017365 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.017287    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"
Jul 24 15:03:50.017365 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.017330    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" event={"ID":"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8","Type":"ContainerDied","Data":"1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f"}
Jul 24 15:03:50.017472 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.017387    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz" event={"ID":"c4dabb32-4fd3-4896-8c5c-6bd74aea48c8","Type":"ContainerDied","Data":"33dadfe7360e7a9dd2156dfafc257017c16f4b2d3f5a371dc7b64fa1a73474e9"}
Jul 24 15:03:50.017472 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.017418    2583 scope.go:117] "RemoveContainer" containerID="1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f"
Jul 24 15:03:50.018872 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.018850    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc" event={"ID":"abb11d3d-4996-4a2b-a242-86340124c09c","Type":"ContainerStarted","Data":"bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07"}
Jul 24 15:03:50.018954 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.018899    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:03:50.018954 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.018925    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc" event={"ID":"abb11d3d-4996-4a2b-a242-86340124c09c","Type":"ContainerStarted","Data":"eefb27568c372ef57fc175348fac8f995b8f1e7030133ef8d8ca1530aca754b9"}
Jul 24 15:03:50.023648 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.023630    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:50.027047 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.027029    2583 scope.go:117] "RemoveContainer" containerID="f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798"
Jul 24 15:03:50.042708 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.042684    2583 scope.go:117] "RemoveContainer" containerID="1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f"
Jul 24 15:03:50.043066 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:03:50.043045    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f\": container with ID starting with 1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f not found: ID does not exist" containerID="1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f"
Jul 24 15:03:50.043171 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.043076    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f"} err="failed to get container status \"1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f\": rpc error: code = NotFound desc = could not find container \"1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f\": container with ID starting with 1a5388d641d41b83aafcc55140c9dc7c69c887ca889a2658be1dbd3df6b4505f not found: ID does not exist"
Jul 24 15:03:50.043171 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.043106    2583 scope.go:117] "RemoveContainer" containerID="f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798"
Jul 24 15:03:50.043428 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:03:50.043410    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798\": container with ID starting with f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798 not found: ID does not exist" containerID="f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798"
Jul 24 15:03:50.043546 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.043437    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798"} err="failed to get container status \"f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798\": rpc error: code = NotFound desc = could not find container \"f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798\": container with ID starting with f58f0aeed25f61b24cb9c8f7d2d5b88393b19e36d57a3c0aa2526ee0ffee2798 not found: ID does not exist"
Jul 24 15:03:50.049627 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.049583    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx" podStartSLOduration=2.049566649 podStartE2EDuration="2.049566649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:50.047615498 +0000 UTC m=+1815.200616223" watchObservedRunningTime="2026-07-24 15:03:50.049566649 +0000 UTC m=+1815.202567357"
Jul 24 15:03:50.075625 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.075570    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc" podStartSLOduration=2.075556419 podStartE2EDuration="2.075556419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:50.074413971 +0000 UTC m=+1815.227414697" watchObservedRunningTime="2026-07-24 15:03:50.075556419 +0000 UTC m=+1815.228557162"
Jul 24 15:03:50.101255 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.101229    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"]
Jul 24 15:03:50.110171 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.110151    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz"]
Jul 24 15:03:50.206431 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:50.206388    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"]
Jul 24 15:03:50.215949 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:03:50.215921    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c14e193_2b4b_478c_824c_fc1208f65c32.slice/crio-34dd684d5f3067fca9e18c3c90176bfc97eba6caff42384e11e218f88d620714 WatchSource:0}: Error finding container 34dd684d5f3067fca9e18c3c90176bfc97eba6caff42384e11e218f88d620714: Status 404 returned error can't find the container with id 34dd684d5f3067fca9e18c3c90176bfc97eba6caff42384e11e218f88d620714
Jul 24 15:03:51.027077 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:51.027033    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs" event={"ID":"3c14e193-2b4b-478c-824c-fc1208f65c32","Type":"ContainerStarted","Data":"b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f"}
Jul 24 15:03:51.027493 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:51.027394    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:03:51.027493 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:51.027435    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs" event={"ID":"3c14e193-2b4b-478c-824c-fc1208f65c32","Type":"ContainerStarted","Data":"34dd684d5f3067fca9e18c3c90176bfc97eba6caff42384e11e218f88d620714"}
Jul 24 15:03:51.049587 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:51.049541    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs" podStartSLOduration=2.694828014 podStartE2EDuration="3.049526804s" totalImagesPullingTime="354.69879ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:51.046473357 +0000 UTC m=+1816.199474082" watchObservedRunningTime="2026-07-24 15:03:51.049526804 +0000 UTC m=+1816.202527529"
Jul 24 15:03:51.343925 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:51.343886    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" path="/var/lib/kubelet/pods/c4dabb32-4fd3-4896-8c5c-6bd74aea48c8/volumes"
Jul 24 15:03:57.037294 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:03:57.037261    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:04:01.029542 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:01.029514    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:04:01.029908 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:01.029584    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:04:23.409154 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.409119    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"]
Jul 24 15:04:23.409770 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.409355    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs" podUID="3c14e193-2b4b-478c-824c-fc1208f65c32" containerName="maas-api" containerID="cri-o://b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f" gracePeriod=30
Jul 24 15:04:23.441599 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.441568    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:23.466903 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.466877    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"]
Jul 24 15:04:23.467117 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.467093    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc" podUID="abb11d3d-4996-4a2b-a242-86340124c09c" containerName="payload-processing" containerID="cri-o://bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07" gracePeriod=30
Jul 24 15:04:23.482294 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.482266    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"]
Jul 24 15:04:23.482512 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.482488    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx" podUID="06d3e63a-fd2b-4082-9f20-14fe3e116bdc" containerName="payload-pre-processing" containerID="cri-o://062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389" gracePeriod=30
Jul 24 15:04:23.539240 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.539213    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:23.756867 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.756840    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:04:23.780804 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.780776    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:04:23.791653 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.791631    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:04:23.791771 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.791703    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ngd4\" (UniqueName: \"kubernetes.io/projected/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-kube-api-access-2ngd4\") pod \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\" (UID: \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\") "
Jul 24 15:04:23.791844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.791791    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-plugins-config-volume\") pod \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\" (UID: \"06d3e63a-fd2b-4082-9f20-14fe3e116bdc\") "
Jul 24 15:04:23.792122 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.792100    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06d3e63a-fd2b-4082-9f20-14fe3e116bdc" (UID: "06d3e63a-fd2b-4082-9f20-14fe3e116bdc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:04:23.793564 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.793542    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-kube-api-access-2ngd4" (OuterVolumeSpecName: "kube-api-access-2ngd4") pod "06d3e63a-fd2b-4082-9f20-14fe3e116bdc" (UID: "06d3e63a-fd2b-4082-9f20-14fe3e116bdc"). InnerVolumeSpecName "kube-api-access-2ngd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:04:23.893150 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.893122    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls\") pod \"3c14e193-2b4b-478c-824c-fc1208f65c32\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") "
Jul 24 15:04:23.893265 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.893198    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb11d3d-4996-4a2b-a242-86340124c09c-plugins-config-volume\") pod \"abb11d3d-4996-4a2b-a242-86340124c09c\" (UID: \"abb11d3d-4996-4a2b-a242-86340124c09c\") "
Jul 24 15:04:23.893265 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.893231    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57fsl\" (UniqueName: \"kubernetes.io/projected/3c14e193-2b4b-478c-824c-fc1208f65c32-kube-api-access-57fsl\") pod \"3c14e193-2b4b-478c-824c-fc1208f65c32\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") "
Jul 24 15:04:23.893265 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.893247    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjn4c\" (UniqueName: \"kubernetes.io/projected/abb11d3d-4996-4a2b-a242-86340124c09c-kube-api-access-qjn4c\") pod \"abb11d3d-4996-4a2b-a242-86340124c09c\" (UID: \"abb11d3d-4996-4a2b-a242-86340124c09c\") "
Jul 24 15:04:23.893427 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.893268    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c14e193-2b4b-478c-824c-fc1208f65c32-openshift-service-ca\") pod \"3c14e193-2b4b-478c-824c-fc1208f65c32\" (UID: \"3c14e193-2b4b-478c-824c-fc1208f65c32\") "
Jul 24 15:04:23.893505 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.893431    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:04:23.893505 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.893477    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ngd4\" (UniqueName: \"kubernetes.io/projected/06d3e63a-fd2b-4082-9f20-14fe3e116bdc-kube-api-access-2ngd4\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:04:23.893661 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.893637    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abb11d3d-4996-4a2b-a242-86340124c09c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abb11d3d-4996-4a2b-a242-86340124c09c" (UID: "abb11d3d-4996-4a2b-a242-86340124c09c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:04:23.893749 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.893727    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c14e193-2b4b-478c-824c-fc1208f65c32-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c14e193-2b4b-478c-824c-fc1208f65c32" (UID: "3c14e193-2b4b-478c-824c-fc1208f65c32"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:04:23.895144 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.895118    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c14e193-2b4b-478c-824c-fc1208f65c32" (UID: "3c14e193-2b4b-478c-824c-fc1208f65c32"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:04:23.895236 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.895179    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb11d3d-4996-4a2b-a242-86340124c09c-kube-api-access-qjn4c" (OuterVolumeSpecName: "kube-api-access-qjn4c") pod "abb11d3d-4996-4a2b-a242-86340124c09c" (UID: "abb11d3d-4996-4a2b-a242-86340124c09c"). InnerVolumeSpecName "kube-api-access-qjn4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:04:23.895236 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.895201    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c14e193-2b4b-478c-824c-fc1208f65c32-kube-api-access-57fsl" (OuterVolumeSpecName: "kube-api-access-57fsl") pod "3c14e193-2b4b-478c-824c-fc1208f65c32" (UID: "3c14e193-2b4b-478c-824c-fc1208f65c32"). InnerVolumeSpecName "kube-api-access-57fsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:04:23.994491 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.994411    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abb11d3d-4996-4a2b-a242-86340124c09c-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:04:23.994491 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.994432    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57fsl\" (UniqueName: \"kubernetes.io/projected/3c14e193-2b4b-478c-824c-fc1208f65c32-kube-api-access-57fsl\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:04:23.994491 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.994441    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjn4c\" (UniqueName: \"kubernetes.io/projected/abb11d3d-4996-4a2b-a242-86340124c09c-kube-api-access-qjn4c\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:04:23.994491 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.994450    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c14e193-2b4b-478c-824c-fc1208f65c32-openshift-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:04:23.994491 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:23.994480    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c14e193-2b4b-478c-824c-fc1208f65c32-maas-api-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.180424 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.180399    2583 generic.go:356] "Generic (PLEG): container finished" podID="abb11d3d-4996-4a2b-a242-86340124c09c" containerID="bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07" exitCode=0
Jul 24 15:04:24.180572 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.180470    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"
Jul 24 15:04:24.180572 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.180491    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc" event={"ID":"abb11d3d-4996-4a2b-a242-86340124c09c","Type":"ContainerDied","Data":"bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07"}
Jul 24 15:04:24.180572 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.180549    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc" event={"ID":"abb11d3d-4996-4a2b-a242-86340124c09c","Type":"ContainerDied","Data":"eefb27568c372ef57fc175348fac8f995b8f1e7030133ef8d8ca1530aca754b9"}
Jul 24 15:04:24.180731 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.180577    2583 scope.go:117] "RemoveContainer" containerID="bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07"
Jul 24 15:04:24.181809 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.181784    2583 generic.go:356] "Generic (PLEG): container finished" podID="3c14e193-2b4b-478c-824c-fc1208f65c32" containerID="b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f" exitCode=0
Jul 24 15:04:24.181908 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.181813    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs" event={"ID":"3c14e193-2b4b-478c-824c-fc1208f65c32","Type":"ContainerDied","Data":"b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f"}
Jul 24 15:04:24.181908 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.181835    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"
Jul 24 15:04:24.181908 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.181871    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs" event={"ID":"3c14e193-2b4b-478c-824c-fc1208f65c32","Type":"ContainerDied","Data":"34dd684d5f3067fca9e18c3c90176bfc97eba6caff42384e11e218f88d620714"}
Jul 24 15:04:24.183413 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.183390    2583 generic.go:356] "Generic (PLEG): container finished" podID="06d3e63a-fd2b-4082-9f20-14fe3e116bdc" containerID="062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389" exitCode=0
Jul 24 15:04:24.183544 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.183430    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"
Jul 24 15:04:24.183544 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.183445    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx" event={"ID":"06d3e63a-fd2b-4082-9f20-14fe3e116bdc","Type":"ContainerDied","Data":"062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389"}
Jul 24 15:04:24.183544 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.183495    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx" event={"ID":"06d3e63a-fd2b-4082-9f20-14fe3e116bdc","Type":"ContainerDied","Data":"421c11a47909010aaeb3044fda8dd1606519bedbe74c3daa39461af54c17a00d"}
Jul 24 15:04:24.189421 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.189392    2583 scope.go:117] "RemoveContainer" containerID="bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07"
Jul 24 15:04:24.189701 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:04:24.189678    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07\": container with ID starting with bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07 not found: ID does not exist" containerID="bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07"
Jul 24 15:04:24.189796 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.189705    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07"} err="failed to get container status \"bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07\": rpc error: code = NotFound desc = could not find container \"bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07\": container with ID starting with bff1ce72a98e85a6905ca144f1b21513fd1033a65c9ad0c72a748f9c85485b07 not found: ID does not exist"
Jul 24 15:04:24.189796 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.189728    2583 scope.go:117] "RemoveContainer" containerID="b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f"
Jul 24 15:04:24.198535 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.198520    2583 scope.go:117] "RemoveContainer" containerID="b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f"
Jul 24 15:04:24.198776 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:04:24.198760    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f\": container with ID starting with b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f not found: ID does not exist" containerID="b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f"
Jul 24 15:04:24.198848 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.198785    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f"} err="failed to get container status \"b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f\": rpc error: code = NotFound desc = could not find container \"b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f\": container with ID starting with b9925f098c44eb52a5059651fa8521220f320e9a333dcda89c16c04b62833a0f not found: ID does not exist"
Jul 24 15:04:24.198848 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.198805    2583 scope.go:117] "RemoveContainer" containerID="062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389"
Jul 24 15:04:24.205924 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.205911    2583 scope.go:117] "RemoveContainer" containerID="062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389"
Jul 24 15:04:24.206152 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:04:24.206136    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389\": container with ID starting with 062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389 not found: ID does not exist" containerID="062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389"
Jul 24 15:04:24.206196 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.206157    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389"} err="failed to get container status \"062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389\": rpc error: code = NotFound desc = could not find container \"062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389\": container with ID starting with 062c49ed979eea3e520eea8ff015106d8079925572e0c1e93cf316f5a62e7389 not found: ID does not exist"
Jul 24 15:04:24.222496 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.222473    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"]
Jul 24 15:04:24.227669 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.227649    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc"]
Jul 24 15:04:24.241200 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.241179    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"]
Jul 24 15:04:24.245656 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.245606    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84b622a4-644dcd745-bkgxs"]
Jul 24 15:04:24.256429 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.256409    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"]
Jul 24 15:04:24.262344 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:24.262320    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx"]
Jul 24 15:04:25.342076 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:25.342046    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06d3e63a-fd2b-4082-9f20-14fe3e116bdc" path="/var/lib/kubelet/pods/06d3e63a-fd2b-4082-9f20-14fe3e116bdc/volumes"
Jul 24 15:04:25.342414 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:25.342404    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c14e193-2b4b-478c-824c-fc1208f65c32" path="/var/lib/kubelet/pods/3c14e193-2b4b-478c-824c-fc1208f65c32/volumes"
Jul 24 15:04:25.342728 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:25.342715    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abb11d3d-4996-4a2b-a242-86340124c09c" path="/var/lib/kubelet/pods/abb11d3d-4996-4a2b-a242-86340124c09c/volumes"
Jul 24 15:04:25.980982 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:25.980947    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:27.978604 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:27.978570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:28.227742 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:28.227705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:28.495691 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:28.495641    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:29.022584 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:29.022548    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:29.111153 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:29.111121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:29.212023 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:29.211998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:30.727983 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:30.727958    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:36.814637 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:36.814604    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:36.920838 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:36.920798    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:37.015793 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:37.015764    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:37.179596 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:37.179563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:37.630490 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:04:37.630422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:03.733759 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:03.733658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:10.688033 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:10.688002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:10.793074 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:10.793045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:10.889811 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:10.889780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:26.869397 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:26.869367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:33.169811 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:33.169774    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:33.232059 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:33.232009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:33.404316 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:33.404281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:33.503949 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:33.503860    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:34.848673 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:34.848627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:43.940655 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:43.940618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:44.792789 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:44.792759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:51.693716 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:51.693676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:51.797608 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:51.797572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:51.893687 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:51.893636    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:51.993085 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:51.993017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:52.960403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:52.960340    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:53.086488 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:53.086432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:54.949356 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:05:54.949313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:01.483693 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:01.483660    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:01.548315 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:01.548288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:02.127865 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:02.127832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:08.727153 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:08.727119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:08.804764 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:08.804732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:08.996301 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:08.996240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:09.320484 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320392    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"]
Jul 24 15:06:09.320788 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320774    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abb11d3d-4996-4a2b-a242-86340124c09c" containerName="payload-processing"
Jul 24 15:06:09.320829 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320789    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb11d3d-4996-4a2b-a242-86340124c09c" containerName="payload-processing"
Jul 24 15:06:09.320829 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320801    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy"
Jul 24 15:06:09.320829 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320811    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy"
Jul 24 15:06:09.320829 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320823    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy"
Jul 24 15:06:09.320829 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320828    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy"
Jul 24 15:06:09.321018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320842    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d3e63a-fd2b-4082-9f20-14fe3e116bdc" containerName="payload-pre-processing"
Jul 24 15:06:09.321018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320847    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d3e63a-fd2b-4082-9f20-14fe3e116bdc" containerName="payload-pre-processing"
Jul 24 15:06:09.321018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320854    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c14e193-2b4b-478c-824c-fc1208f65c32" containerName="maas-api"
Jul 24 15:06:09.321018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320859    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c14e193-2b4b-478c-824c-fc1208f65c32" containerName="maas-api"
Jul 24 15:06:09.321018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320920    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="abb11d3d-4996-4a2b-a242-86340124c09c" containerName="payload-processing"
Jul 24 15:06:09.321018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320927    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy"
Jul 24 15:06:09.321018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320934    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c14e193-2b4b-478c-824c-fc1208f65c32" containerName="maas-api"
Jul 24 15:06:09.321018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320940    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="06d3e63a-fd2b-4082-9f20-14fe3e116bdc" containerName="payload-pre-processing"
Jul 24 15:06:09.321018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.320946    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4dabb32-4fd3-4896-8c5c-6bd74aea48c8" containerName="istio-proxy"
Jul 24 15:06:09.323841 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.323825    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.326619 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.326600    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c811f4d5-openshift-default-dockercfg-z5mqj\""
Jul 24 15:06:09.350857 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.350832    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"]
Jul 24 15:06:09.358007 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.357981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:09.404569 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.404548    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkm8m\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-kube-api-access-dkm8m\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.404670 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.404580    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-certs\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.404670 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.404600    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-podinfo\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.404670 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.404619    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-envoy\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.404775 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.404707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-data\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.404775 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.404743    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-socket\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.404840 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.404795    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-credential-socket\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.404840 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.404816    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istiod-ca-cert\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.404902 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.404843    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-token\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.424246 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.424226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:09.505736 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.505704    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-data\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.505928 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.505910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-socket\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506010 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.505993    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-credential-socket\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506030    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istiod-ca-cert\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506096 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506063    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-token\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506140 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkm8m\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-kube-api-access-dkm8m\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506192 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506168    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-data\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506244 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506195    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-socket\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506244 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-credential-socket\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506351 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506275    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-certs\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506351 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506306    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-podinfo\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506351 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-envoy\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506655 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506636    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-certs\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.506899 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.506879    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istiod-ca-cert\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.508384 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.508363    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-envoy\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.508628 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.508609    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-podinfo\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.515638 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.515616    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkm8m\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-kube-api-access-dkm8m\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.515742 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.515711    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-token\") pod \"e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") " pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.634449 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.634422    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:09.772901 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.772876    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"]
Jul 24 15:06:09.774737 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:06:09.774710    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2216c91f_57d8_48cb_a4e3_6bc0433702e4.slice/crio-19ca64d9cc6ecb3c8c64cd8a3382bd1a16de1439c2b5e7f33bc024998a2da187 WatchSource:0}: Error finding container 19ca64d9cc6ecb3c8c64cd8a3382bd1a16de1439c2b5e7f33bc024998a2da187: Status 404 returned error can't find the container with id 19ca64d9cc6ecb3c8c64cd8a3382bd1a16de1439c2b5e7f33bc024998a2da187
Jul 24 15:06:09.776834 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.776801    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:06:09.776924 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.776872    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:06:09.776924 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:09.776900    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:06:10.663417 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:10.663375    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" event={"ID":"2216c91f-57d8-48cb-a4e3-6bc0433702e4","Type":"ContainerStarted","Data":"b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8"}
Jul 24 15:06:10.663621 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:10.663478    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" event={"ID":"2216c91f-57d8-48cb-a4e3-6bc0433702e4","Type":"ContainerStarted","Data":"19ca64d9cc6ecb3c8c64cd8a3382bd1a16de1439c2b5e7f33bc024998a2da187"}
Jul 24 15:06:10.685179 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:10.685109    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" podStartSLOduration=1.6850964240000001 podStartE2EDuration="1.685096424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:06:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:06:10.68355179 +0000 UTC m=+1955.836552516" watchObservedRunningTime="2026-07-24 15:06:10.685096424 +0000 UTC m=+1955.838097204"
Jul 24 15:06:11.635121 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:11.635081    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:11.639894 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:11.639868    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:11.667770 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:11.667748    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:11.668407 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:11.668388    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:12.701769 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.701691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:12.813215 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.813187    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"]
Jul 24 15:06:12.816624 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.816607    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:12.819896 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.819866    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c811f4d5-serving-cert\""
Jul 24 15:06:12.832541 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.832441    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:12.834637 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.834615    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"]
Jul 24 15:06:12.933195 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.933166    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-maas-api-tls\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:12.933352 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.933199    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-openshift-service-ca\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:12.933352 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.933284    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6bjd\" (UniqueName: \"kubernetes.io/projected/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-kube-api-access-q6bjd\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:12.945178 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:12.945149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:13.034015 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.033953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-maas-api-tls\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:13.034015 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.033983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-openshift-service-ca\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:13.034238 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.034042    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6bjd\" (UniqueName: \"kubernetes.io/projected/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-kube-api-access-q6bjd\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:13.034238 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:06:13.034095    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-serving-cert: secret "maas-api-e2e-mt-c811f4d5-serving-cert" not found
Jul 24 15:06:13.034238 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:06:13.034174    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-maas-api-tls podName:c1d04e06-0b7c-4f84-a19c-9a3171b6119d nodeName:}" failed. No retries permitted until 2026-07-24 15:06:13.534152817 +0000 UTC m=+1958.687153521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-maas-api-tls") pod "maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx" (UID: "c1d04e06-0b7c-4f84-a19c-9a3171b6119d") : secret "maas-api-e2e-mt-c811f4d5-serving-cert" not found
Jul 24 15:06:13.034844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.034822    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-openshift-service-ca\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:13.046950 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.046930    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6bjd\" (UniqueName: \"kubernetes.io/projected/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-kube-api-access-q6bjd\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:13.188802 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.188779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:13.537307 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.537270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-maas-api-tls\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:13.539643 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.539618    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-maas-api-tls\") pod \"maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:13.665637 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.665604    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:13.726140 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.726107    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:13.760816 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.760785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:13.865195 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.865164    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"]
Jul 24 15:06:13.869732 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.869710    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"]
Jul 24 15:06:13.869880 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.869865    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:06:13.871398 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:06:13.871375    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d04e06_0b7c_4f84_a19c_9a3171b6119d.slice/crio-21d47b2155d6fe8652f26ee6c88ac03c3b6eac89d6bbd8cea6903fdaddffcc54 WatchSource:0}: Error finding container 21d47b2155d6fe8652f26ee6c88ac03c3b6eac89d6bbd8cea6903fdaddffcc54: Status 404 returned error can't find the container with id 21d47b2155d6fe8652f26ee6c88ac03c3b6eac89d6bbd8cea6903fdaddffcc54
Jul 24 15:06:13.872481 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.872435    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c811f4d5\""
Jul 24 15:06:13.872569 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.872549    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c811f4d5-dockercfg-9jbws\""
Jul 24 15:06:13.878363 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.878341    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"]
Jul 24 15:06:13.943142 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.943111    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"]
Jul 24 15:06:13.947231 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.947208    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:13.957123 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:13.957097    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"]
Jul 24 15:06:14.043703 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.043675    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11a3afe6-48a8-4047-a18f-a852e6580910-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7\" (UID: \"11a3afe6-48a8-4047-a18f-a852e6580910\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:06:14.043828 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.043730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2c3e7e3-bb05-4817-816d-d414d103a69b-plugins-config-volume\") pod \"payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg\" (UID: \"b2c3e7e3-bb05-4817-816d-d414d103a69b\") " pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:14.043828 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.043792    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vstl\" (UniqueName: \"kubernetes.io/projected/11a3afe6-48a8-4047-a18f-a852e6580910-kube-api-access-9vstl\") pod \"payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7\" (UID: \"11a3afe6-48a8-4047-a18f-a852e6580910\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:06:14.043828 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.043821    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6bwk\" (UniqueName: \"kubernetes.io/projected/b2c3e7e3-bb05-4817-816d-d414d103a69b-kube-api-access-b6bwk\") pod \"payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg\" (UID: \"b2c3e7e3-bb05-4817-816d-d414d103a69b\") " pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:14.144540 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.144507    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11a3afe6-48a8-4047-a18f-a852e6580910-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7\" (UID: \"11a3afe6-48a8-4047-a18f-a852e6580910\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:06:14.144698 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.144571    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2c3e7e3-bb05-4817-816d-d414d103a69b-plugins-config-volume\") pod \"payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg\" (UID: \"b2c3e7e3-bb05-4817-816d-d414d103a69b\") " pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:14.144698 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.144661    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vstl\" (UniqueName: \"kubernetes.io/projected/11a3afe6-48a8-4047-a18f-a852e6580910-kube-api-access-9vstl\") pod \"payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7\" (UID: \"11a3afe6-48a8-4047-a18f-a852e6580910\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:06:14.144863 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.144698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6bwk\" (UniqueName: \"kubernetes.io/projected/b2c3e7e3-bb05-4817-816d-d414d103a69b-kube-api-access-b6bwk\") pod \"payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg\" (UID: \"b2c3e7e3-bb05-4817-816d-d414d103a69b\") " pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:14.145380 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.145354    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11a3afe6-48a8-4047-a18f-a852e6580910-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7\" (UID: \"11a3afe6-48a8-4047-a18f-a852e6580910\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:06:14.145487 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.145359    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2c3e7e3-bb05-4817-816d-d414d103a69b-plugins-config-volume\") pod \"payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg\" (UID: \"b2c3e7e3-bb05-4817-816d-d414d103a69b\") " pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:14.160129 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.160103    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6bwk\" (UniqueName: \"kubernetes.io/projected/b2c3e7e3-bb05-4817-816d-d414d103a69b-kube-api-access-b6bwk\") pod \"payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg\" (UID: \"b2c3e7e3-bb05-4817-816d-d414d103a69b\") " pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:14.160241 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.160171    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vstl\" (UniqueName: \"kubernetes.io/projected/11a3afe6-48a8-4047-a18f-a852e6580910-kube-api-access-9vstl\") pod \"payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7\" (UID: \"11a3afe6-48a8-4047-a18f-a852e6580910\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:06:14.188600 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.188575    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:06:14.259386 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.259357    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:14.324939 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:06:14.324888    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a3afe6_48a8_4047_a18f_a852e6580910.slice/crio-dd328c2d0d2a08f031df52d96ff1c2d02a58365b1e3f89249ed0e276f7144a34 WatchSource:0}: Error finding container dd328c2d0d2a08f031df52d96ff1c2d02a58365b1e3f89249ed0e276f7144a34: Status 404 returned error can't find the container with id dd328c2d0d2a08f031df52d96ff1c2d02a58365b1e3f89249ed0e276f7144a34
Jul 24 15:06:14.325557 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.325156    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"]
Jul 24 15:06:14.401369 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.401348    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"]
Jul 24 15:06:14.402925 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:06:14.402900    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c3e7e3_bb05_4817_816d_d414d103a69b.slice/crio-2aa3e978951410e6b71a2d2eef04f0fedb97aaf379c2ef5887ae3594619b5b5c WatchSource:0}: Error finding container 2aa3e978951410e6b71a2d2eef04f0fedb97aaf379c2ef5887ae3594619b5b5c: Status 404 returned error can't find the container with id 2aa3e978951410e6b71a2d2eef04f0fedb97aaf379c2ef5887ae3594619b5b5c
Jul 24 15:06:14.684388 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.684296    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx" event={"ID":"c1d04e06-0b7c-4f84-a19c-9a3171b6119d","Type":"ContainerStarted","Data":"baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0"}
Jul 24 15:06:14.684388 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.684373    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:14.684632 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.684403    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx" event={"ID":"c1d04e06-0b7c-4f84-a19c-9a3171b6119d","Type":"ContainerStarted","Data":"21d47b2155d6fe8652f26ee6c88ac03c3b6eac89d6bbd8cea6903fdaddffcc54"}
Jul 24 15:06:14.685846 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.685819    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7" event={"ID":"11a3afe6-48a8-4047-a18f-a852e6580910","Type":"ContainerStarted","Data":"9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4"}
Jul 24 15:06:14.685984 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.685861    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7" event={"ID":"11a3afe6-48a8-4047-a18f-a852e6580910","Type":"ContainerStarted","Data":"dd328c2d0d2a08f031df52d96ff1c2d02a58365b1e3f89249ed0e276f7144a34"}
Jul 24 15:06:14.685984 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.685916    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:06:14.687324 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.687305    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg" event={"ID":"b2c3e7e3-bb05-4817-816d-d414d103a69b","Type":"ContainerStarted","Data":"f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13"}
Jul 24 15:06:14.687431 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.687344    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:14.687431 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.687360    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg" event={"ID":"b2c3e7e3-bb05-4817-816d-d414d103a69b","Type":"ContainerStarted","Data":"2aa3e978951410e6b71a2d2eef04f0fedb97aaf379c2ef5887ae3594619b5b5c"}
Jul 24 15:06:14.706490 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.706432    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx" podStartSLOduration=2.115147596 podStartE2EDuration="2.706420052s" totalImagesPullingTime="591.272456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:06:14.704787258 +0000 UTC m=+1959.857787985" watchObservedRunningTime="2026-07-24 15:06:14.706420052 +0000 UTC m=+1959.859420777"
Jul 24 15:06:14.751937 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.751882    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7" podStartSLOduration=1.751863246 podStartE2EDuration="1.751863246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:06:14.74407448 +0000 UTC m=+1959.897075207" watchObservedRunningTime="2026-07-24 15:06:14.751863246 +0000 UTC m=+1959.904863976"
Jul 24 15:06:14.778727 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:14.778689    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg" podStartSLOduration=1.7786659 podStartE2EDuration="1.7786659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:06:14.778142448 +0000 UTC m=+1959.931143173" watchObservedRunningTime="2026-07-24 15:06:14.7786659 +0000 UTC m=+1959.931666618"
Jul 24 15:06:17.206239 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:17.206208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:20.697964 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:20.697936    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:06:23.731134 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:23.731106    2583 generic.go:356] "Generic (PLEG): container finished" podID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerID="b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8" exitCode=0
Jul 24 15:06:23.731551 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:23.731153    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" event={"ID":"2216c91f-57d8-48cb-a4e3-6bc0433702e4","Type":"ContainerDied","Data":"b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8"}
Jul 24 15:06:23.731631 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:23.731557    2583 scope.go:117] "RemoveContainer" containerID="b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8"
Jul 24 15:06:23.732704 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:23.732665    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:06:23.732795 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:23.732735    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:06:23.732795 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:23.732770    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:06:24.738991 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:24.738952    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" event={"ID":"2216c91f-57d8-48cb-a4e3-6bc0433702e4","Type":"ContainerStarted","Data":"1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee"}
Jul 24 15:06:25.636182 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:25.636148    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:25.636581 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:25.636554    2583 patch_prober.go:28] interesting pod/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 24 15:06:25.636745 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:25.636721    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 24 15:06:25.693506 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:25.693465    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7" podUID="11a3afe6-48a8-4047-a18f-a852e6580910" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.67:9004: connect: connection refused"
Jul 24 15:06:25.693921 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:25.693899    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:06:26.635709 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:26.635666    2583 patch_prober.go:28] interesting pod/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 24 15:06:26.636161 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:26.635762    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 24 15:06:26.668323 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:26.668290    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:27.634870 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:27.634831    2583 patch_prober.go:28] interesting pod/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 24 15:06:27.635071 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:27.634891    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 24 15:06:28.635599 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:28.635557    2583 patch_prober.go:28] interesting pod/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 24 15:06:28.636043 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:28.635625    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 24 15:06:29.639250 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:29.639219    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:29.640323 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:29.640302    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:06:34.189498 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:34.189438    2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7" podUID="11a3afe6-48a8-4047-a18f-a852e6580910" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.67:9004: connect: connection refused"
Jul 24 15:06:35.693427 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:06:35.693400    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:07:44.218785 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:07:44.218751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:44.318908 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:07:44.318881    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:31.727187 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:31.727156    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"]
Jul 24 15:08:31.727703 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:31.727353    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx" podUID="c1d04e06-0b7c-4f84-a19c-9a3171b6119d" containerName="maas-api" containerID="cri-o://baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0" gracePeriod=30
Jul 24 15:08:31.770182 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:31.770157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:31.804617 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:31.804487    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"]
Jul 24 15:08:31.804752 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:31.804705    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg" podUID="b2c3e7e3-bb05-4817-816d-d414d103a69b" containerName="payload-processing" containerID="cri-o://f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13" gracePeriod=30
Jul 24 15:08:31.809819 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:31.809796    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"]
Jul 24 15:08:31.809979 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:31.809960    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7" podUID="11a3afe6-48a8-4047-a18f-a852e6580910" containerName="payload-pre-processing" containerID="cri-o://9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4" gracePeriod=30
Jul 24 15:08:31.870714 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:31.870674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:31.967750 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:31.967704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:32.069428 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.069398    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:08:32.073589 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.073571    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:08:32.108159 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.108133    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:08:32.184313 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.184287    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-openshift-service-ca\") pod \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") "
Jul 24 15:08:32.184442 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.184332    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vstl\" (UniqueName: \"kubernetes.io/projected/11a3afe6-48a8-4047-a18f-a852e6580910-kube-api-access-9vstl\") pod \"11a3afe6-48a8-4047-a18f-a852e6580910\" (UID: \"11a3afe6-48a8-4047-a18f-a852e6580910\") "
Jul 24 15:08:32.184442 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.184368    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6bwk\" (UniqueName: \"kubernetes.io/projected/b2c3e7e3-bb05-4817-816d-d414d103a69b-kube-api-access-b6bwk\") pod \"b2c3e7e3-bb05-4817-816d-d414d103a69b\" (UID: \"b2c3e7e3-bb05-4817-816d-d414d103a69b\") "
Jul 24 15:08:32.184442 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.184384    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6bjd\" (UniqueName: \"kubernetes.io/projected/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-kube-api-access-q6bjd\") pod \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") "
Jul 24 15:08:32.184442 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.184412    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-maas-api-tls\") pod \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\" (UID: \"c1d04e06-0b7c-4f84-a19c-9a3171b6119d\") "
Jul 24 15:08:32.184442 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.184439    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2c3e7e3-bb05-4817-816d-d414d103a69b-plugins-config-volume\") pod \"b2c3e7e3-bb05-4817-816d-d414d103a69b\" (UID: \"b2c3e7e3-bb05-4817-816d-d414d103a69b\") "
Jul 24 15:08:32.184715 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.184498    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11a3afe6-48a8-4047-a18f-a852e6580910-plugins-config-volume\") pod \"11a3afe6-48a8-4047-a18f-a852e6580910\" (UID: \"11a3afe6-48a8-4047-a18f-a852e6580910\") "
Jul 24 15:08:32.184773 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.184730    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1d04e06-0b7c-4f84-a19c-9a3171b6119d" (UID: "c1d04e06-0b7c-4f84-a19c-9a3171b6119d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:08:32.184862 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.184828    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2c3e7e3-bb05-4817-816d-d414d103a69b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2c3e7e3-bb05-4817-816d-d414d103a69b" (UID: "b2c3e7e3-bb05-4817-816d-d414d103a69b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:08:32.185155 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.185114    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a3afe6-48a8-4047-a18f-a852e6580910-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11a3afe6-48a8-4047-a18f-a852e6580910" (UID: "11a3afe6-48a8-4047-a18f-a852e6580910"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:08:32.186505 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.186478    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1d04e06-0b7c-4f84-a19c-9a3171b6119d" (UID: "c1d04e06-0b7c-4f84-a19c-9a3171b6119d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:08:32.186590 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.186482    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a3afe6-48a8-4047-a18f-a852e6580910-kube-api-access-9vstl" (OuterVolumeSpecName: "kube-api-access-9vstl") pod "11a3afe6-48a8-4047-a18f-a852e6580910" (UID: "11a3afe6-48a8-4047-a18f-a852e6580910"). InnerVolumeSpecName "kube-api-access-9vstl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:08:32.186590 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.186532    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-kube-api-access-q6bjd" (OuterVolumeSpecName: "kube-api-access-q6bjd") pod "c1d04e06-0b7c-4f84-a19c-9a3171b6119d" (UID: "c1d04e06-0b7c-4f84-a19c-9a3171b6119d"). InnerVolumeSpecName "kube-api-access-q6bjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:08:32.186590 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.186534    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c3e7e3-bb05-4817-816d-d414d103a69b-kube-api-access-b6bwk" (OuterVolumeSpecName: "kube-api-access-b6bwk") pod "b2c3e7e3-bb05-4817-816d-d414d103a69b" (UID: "b2c3e7e3-bb05-4817-816d-d414d103a69b"). InnerVolumeSpecName "kube-api-access-b6bwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:08:32.285593 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.285540    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-openshift-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:32.285593 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.285560    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vstl\" (UniqueName: \"kubernetes.io/projected/11a3afe6-48a8-4047-a18f-a852e6580910-kube-api-access-9vstl\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:32.285593 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.285569    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6bwk\" (UniqueName: \"kubernetes.io/projected/b2c3e7e3-bb05-4817-816d-d414d103a69b-kube-api-access-b6bwk\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:32.285593 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.285577    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6bjd\" (UniqueName: \"kubernetes.io/projected/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-kube-api-access-q6bjd\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:32.285593 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.285585    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1d04e06-0b7c-4f84-a19c-9a3171b6119d-maas-api-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:32.285593 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.285593    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2c3e7e3-bb05-4817-816d-d414d103a69b-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:32.285814 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.285601    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11a3afe6-48a8-4047-a18f-a852e6580910-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:32.339429 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.339409    2583 generic.go:356] "Generic (PLEG): container finished" podID="c1d04e06-0b7c-4f84-a19c-9a3171b6119d" containerID="baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0" exitCode=0
Jul 24 15:08:32.339540 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.339495    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"
Jul 24 15:08:32.339603 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.339494    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx" event={"ID":"c1d04e06-0b7c-4f84-a19c-9a3171b6119d","Type":"ContainerDied","Data":"baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0"}
Jul 24 15:08:32.339660 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.339613    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx" event={"ID":"c1d04e06-0b7c-4f84-a19c-9a3171b6119d","Type":"ContainerDied","Data":"21d47b2155d6fe8652f26ee6c88ac03c3b6eac89d6bbd8cea6903fdaddffcc54"}
Jul 24 15:08:32.339660 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.339647    2583 scope.go:117] "RemoveContainer" containerID="baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0"
Jul 24 15:08:32.340936 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.340912    2583 generic.go:356] "Generic (PLEG): container finished" podID="11a3afe6-48a8-4047-a18f-a852e6580910" containerID="9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4" exitCode=0
Jul 24 15:08:32.341053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.340957    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"
Jul 24 15:08:32.341053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.341005    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7" event={"ID":"11a3afe6-48a8-4047-a18f-a852e6580910","Type":"ContainerDied","Data":"9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4"}
Jul 24 15:08:32.341164 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.341084    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7" event={"ID":"11a3afe6-48a8-4047-a18f-a852e6580910","Type":"ContainerDied","Data":"dd328c2d0d2a08f031df52d96ff1c2d02a58365b1e3f89249ed0e276f7144a34"}
Jul 24 15:08:32.342537 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.342514    2583 generic.go:356] "Generic (PLEG): container finished" podID="b2c3e7e3-bb05-4817-816d-d414d103a69b" containerID="f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13" exitCode=0
Jul 24 15:08:32.342637 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.342571    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg" event={"ID":"b2c3e7e3-bb05-4817-816d-d414d103a69b","Type":"ContainerDied","Data":"f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13"}
Jul 24 15:08:32.342637 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.342576    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"
Jul 24 15:08:32.342637 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.342603    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg" event={"ID":"b2c3e7e3-bb05-4817-816d-d414d103a69b","Type":"ContainerDied","Data":"2aa3e978951410e6b71a2d2eef04f0fedb97aaf379c2ef5887ae3594619b5b5c"}
Jul 24 15:08:32.350326 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.350306    2583 scope.go:117] "RemoveContainer" containerID="baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0"
Jul 24 15:08:32.350638 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:08:32.350619    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0\": container with ID starting with baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0 not found: ID does not exist" containerID="baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0"
Jul 24 15:08:32.350722 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.350645    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0"} err="failed to get container status \"baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0\": rpc error: code = NotFound desc = could not find container \"baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0\": container with ID starting with baa1cee8864fd1641590033c5770cc43ac5ba79223d6cdfd2481cd5eb53dadc0 not found: ID does not exist"
Jul 24 15:08:32.350722 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.350668    2583 scope.go:117] "RemoveContainer" containerID="9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4"
Jul 24 15:08:32.358213 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.358197    2583 scope.go:117] "RemoveContainer" containerID="9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4"
Jul 24 15:08:32.358449 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:08:32.358431    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4\": container with ID starting with 9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4 not found: ID does not exist" containerID="9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4"
Jul 24 15:08:32.358513 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.358468    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4"} err="failed to get container status \"9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4\": rpc error: code = NotFound desc = could not find container \"9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4\": container with ID starting with 9987a5f64f6150d4ac962fc05a6eb85100f5efe85392d290cc7d96d19b15b2d4 not found: ID does not exist"
Jul 24 15:08:32.358513 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.358485    2583 scope.go:117] "RemoveContainer" containerID="f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13"
Jul 24 15:08:32.364027 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.364008    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"]
Jul 24 15:08:32.365556 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.365543    2583 scope.go:117] "RemoveContainer" containerID="f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13"
Jul 24 15:08:32.365778 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:08:32.365760    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13\": container with ID starting with f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13 not found: ID does not exist" containerID="f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13"
Jul 24 15:08:32.365821 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.365786    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13"} err="failed to get container status \"f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13\": rpc error: code = NotFound desc = could not find container \"f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13\": container with ID starting with f7f9c8ae631540c55597ee6c7ec13dbc8e12a32585ff5172635f295f09e45d13 not found: ID does not exist"
Jul 24 15:08:32.369909 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.369890    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c811f4d5-7b66f6c97d-t9ntx"]
Jul 24 15:08:32.380505 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.380484    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"]
Jul 24 15:08:32.383812 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.383792    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7"]
Jul 24 15:08:32.397191 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.397162    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"]
Jul 24 15:08:32.398799 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:32.398778    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg"]
Jul 24 15:08:33.343510 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:33.343476    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11a3afe6-48a8-4047-a18f-a852e6580910" path="/var/lib/kubelet/pods/11a3afe6-48a8-4047-a18f-a852e6580910/volumes"
Jul 24 15:08:33.343989 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:33.343972    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2c3e7e3-bb05-4817-816d-d414d103a69b" path="/var/lib/kubelet/pods/b2c3e7e3-bb05-4817-816d-d414d103a69b/volumes"
Jul 24 15:08:33.344449 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:33.344432    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1d04e06-0b7c-4f84-a19c-9a3171b6119d" path="/var/lib/kubelet/pods/c1d04e06-0b7c-4f84-a19c-9a3171b6119d/volumes"
Jul 24 15:08:35.822436 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:35.822406    2583 scope.go:117] "RemoveContainer" containerID="37eae44be48bf55b73f6d979937b48edf5fcb3b7af58fc4c11f02fc505301cd2"
Jul 24 15:08:36.200542 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:36.200511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:36.610221 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:36.610193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:36.962127 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:36.962064    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:42.776768 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:42.776734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:42.852435 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:42.852407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:43.076192 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:43.076113    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"]
Jul 24 15:08:43.076346 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:43.076299    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy" containerID="cri-o://1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee" gracePeriod=30
Jul 24 15:08:43.165396 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:43.165348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:43.347066 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:43.347023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:43.567099 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:43.567071    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:43.664548 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:43.664476    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:44.065100 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:44.065023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:44.641788 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:44.641752    2583 patch_prober.go:28] interesting pod/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 15:08:44.641949 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:44.641822    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 15:08:48.324933 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.324907    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:08:48.404389 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404365    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-podinfo\") pod \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") "
Jul 24 15:08:48.404624 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404397    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkm8m\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-kube-api-access-dkm8m\") pod \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") "
Jul 24 15:08:48.404624 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404419    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-data\") pod \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") "
Jul 24 15:08:48.404624 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404438    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-envoy\") pod \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") "
Jul 24 15:08:48.404804 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404671    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-credential-socket\") pod \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") "
Jul 24 15:08:48.404804 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404740    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-certs\") pod \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") "
Jul 24 15:08:48.404804 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404769    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istiod-ca-cert\") pod \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") "
Jul 24 15:08:48.405245 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404847    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-socket\") pod \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") "
Jul 24 15:08:48.405245 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404871    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-token\") pod \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\" (UID: \"2216c91f-57d8-48cb-a4e3-6bc0433702e4\") "
Jul 24 15:08:48.405245 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.404907    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2216c91f-57d8-48cb-a4e3-6bc0433702e4" (UID: "2216c91f-57d8-48cb-a4e3-6bc0433702e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:08:48.405245 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.405092    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2216c91f-57d8-48cb-a4e3-6bc0433702e4" (UID: "2216c91f-57d8-48cb-a4e3-6bc0433702e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:08:48.405245 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.405111    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:48.405245 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.405154    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2216c91f-57d8-48cb-a4e3-6bc0433702e4" (UID: "2216c91f-57d8-48cb-a4e3-6bc0433702e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:08:48.405245 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.405218    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2216c91f-57d8-48cb-a4e3-6bc0433702e4" (UID: "2216c91f-57d8-48cb-a4e3-6bc0433702e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:08:48.405648 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.405629    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "2216c91f-57d8-48cb-a4e3-6bc0433702e4" (UID: "2216c91f-57d8-48cb-a4e3-6bc0433702e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:08:48.406639 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.406613    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2216c91f-57d8-48cb-a4e3-6bc0433702e4" (UID: "2216c91f-57d8-48cb-a4e3-6bc0433702e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:08:48.406780 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.406762    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "2216c91f-57d8-48cb-a4e3-6bc0433702e4" (UID: "2216c91f-57d8-48cb-a4e3-6bc0433702e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:08:48.406872 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.406850    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-kube-api-access-dkm8m" (OuterVolumeSpecName: "kube-api-access-dkm8m") pod "2216c91f-57d8-48cb-a4e3-6bc0433702e4" (UID: "2216c91f-57d8-48cb-a4e3-6bc0433702e4"). InnerVolumeSpecName "kube-api-access-dkm8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:08:48.406920 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.406856    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2216c91f-57d8-48cb-a4e3-6bc0433702e4" (UID: "2216c91f-57d8-48cb-a4e3-6bc0433702e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:08:48.417018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.416996    2583 generic.go:356] "Generic (PLEG): container finished" podID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerID="1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee" exitCode=0
Jul 24 15:08:48.417115 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.417060    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"
Jul 24 15:08:48.417179 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.417065    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" event={"ID":"2216c91f-57d8-48cb-a4e3-6bc0433702e4","Type":"ContainerDied","Data":"1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee"}
Jul 24 15:08:48.417179 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.417167    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x" event={"ID":"2216c91f-57d8-48cb-a4e3-6bc0433702e4","Type":"ContainerDied","Data":"19ca64d9cc6ecb3c8c64cd8a3382bd1a16de1439c2b5e7f33bc024998a2da187"}
Jul 24 15:08:48.417280 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.417188    2583 scope.go:117] "RemoveContainer" containerID="1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee"
Jul 24 15:08:48.428373 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.428359    2583 scope.go:117] "RemoveContainer" containerID="b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8"
Jul 24 15:08:48.436534 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.436517    2583 scope.go:117] "RemoveContainer" containerID="1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee"
Jul 24 15:08:48.436789 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:08:48.436770    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee\": container with ID starting with 1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee not found: ID does not exist" containerID="1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee"
Jul 24 15:08:48.436836 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.436796    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee"} err="failed to get container status \"1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee\": rpc error: code = NotFound desc = could not find container \"1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee\": container with ID starting with 1b8563f67999496c8b347774e931c26f32e12bad89540d7e124905a1db5422ee not found: ID does not exist"
Jul 24 15:08:48.436836 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.436812    2583 scope.go:117] "RemoveContainer" containerID="b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8"
Jul 24 15:08:48.437053 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:08:48.437026    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8\": container with ID starting with b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8 not found: ID does not exist" containerID="b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8"
Jul 24 15:08:48.437144 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.437055    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8"} err="failed to get container status \"b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8\": rpc error: code = NotFound desc = could not find container \"b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8\": container with ID starting with b400cc77792af738bfefd0df3aa36a7ae1990bd657d78b9236f76149b49e81c8 not found: ID does not exist"
Jul 24 15:08:48.439508 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.439488    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"]
Jul 24 15:08:48.448047 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.448027    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x"]
Jul 24 15:08:48.505430 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.505409    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:48.505430 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.505428    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:48.505581 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.505436    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:48.505581 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.505445    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:48.505581 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.505471    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:48.505581 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.505479    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkm8m\" (UniqueName: \"kubernetes.io/projected/2216c91f-57d8-48cb-a4e3-6bc0433702e4-kube-api-access-dkm8m\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:48.505581 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.505487    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:48.505581 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:48.505494    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2216c91f-57d8-48cb-a4e3-6bc0433702e4-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:08:49.344162 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:49.344137    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" path="/var/lib/kubelet/pods/2216c91f-57d8-48cb-a4e3-6bc0433702e4/volumes"
Jul 24 15:08:50.172269 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:50.172244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:50.290790 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:50.290763    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:50.395737 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:50.395711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:50.545298 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:50.545231    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:56.844407 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:56.844377    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:56.987630 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:08:56.987606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:03.709823 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:03.709787    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:04.025178 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:04.025096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:04.557249 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:04.557221    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:07.620799 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.620761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:07.763477 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763418    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"]
Jul 24 15:09:07.763783 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763768    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1d04e06-0b7c-4f84-a19c-9a3171b6119d" containerName="maas-api"
Jul 24 15:09:07.763783 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763783    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1d04e06-0b7c-4f84-a19c-9a3171b6119d" containerName="maas-api"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763790    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763796    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763819    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c3e7e3-bb05-4817-816d-d414d103a69b" containerName="payload-processing"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763825    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c3e7e3-bb05-4817-816d-d414d103a69b" containerName="payload-processing"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763831    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a3afe6-48a8-4047-a18f-a852e6580910" containerName="payload-pre-processing"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763837    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a3afe6-48a8-4047-a18f-a852e6580910" containerName="payload-pre-processing"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763899    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a3afe6-48a8-4047-a18f-a852e6580910" containerName="payload-pre-processing"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763909    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763914    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1d04e06-0b7c-4f84-a19c-9a3171b6119d" containerName="maas-api"
Jul 24 15:09:07.763953 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.763919    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c3e7e3-bb05-4817-816d-d414d103a69b" containerName="payload-processing"
Jul 24 15:09:07.768220 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.768201    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:07.771821 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.771798    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:07.772355 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.772333    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e61f3ca0-serving-cert\""
Jul 24 15:09:07.803939 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.803918    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"]
Jul 24 15:09:07.851511 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.851487    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18b380a7-3345-4742-a6ce-78591805e274-maas-api-tls\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:07.851609 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.851523    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b380a7-3345-4742-a6ce-78591805e274-openshift-service-ca\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:07.851675 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.851654    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpccw\" (UniqueName: \"kubernetes.io/projected/18b380a7-3345-4742-a6ce-78591805e274-kube-api-access-zpccw\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:07.919873 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.919806    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:07.952929 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.952901    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpccw\" (UniqueName: \"kubernetes.io/projected/18b380a7-3345-4742-a6ce-78591805e274-kube-api-access-zpccw\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:07.953020 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.952945    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18b380a7-3345-4742-a6ce-78591805e274-maas-api-tls\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:07.953020 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.952978    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b380a7-3345-4742-a6ce-78591805e274-openshift-service-ca\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:07.953112 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:09:07.953094    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-serving-cert: secret "maas-api-e2e-mt-e61f3ca0-serving-cert" not found
Jul 24 15:09:07.953179 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:09:07.953169    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18b380a7-3345-4742-a6ce-78591805e274-maas-api-tls podName:18b380a7-3345-4742-a6ce-78591805e274 nodeName:}" failed. No retries permitted until 2026-07-24 15:09:08.453151825 +0000 UTC m=+2133.606152529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/18b380a7-3345-4742-a6ce-78591805e274-maas-api-tls") pod "maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj" (UID: "18b380a7-3345-4742-a6ce-78591805e274") : secret "maas-api-e2e-mt-e61f3ca0-serving-cert" not found
Jul 24 15:09:07.953664 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.953646    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b380a7-3345-4742-a6ce-78591805e274-openshift-service-ca\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:07.963716 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:07.963698    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpccw\" (UniqueName: \"kubernetes.io/projected/18b380a7-3345-4742-a6ce-78591805e274-kube-api-access-zpccw\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:08.133882 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.133859    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:08.457269 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.457230    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18b380a7-3345-4742-a6ce-78591805e274-maas-api-tls\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:08.459595 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.459575    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18b380a7-3345-4742-a6ce-78591805e274-maas-api-tls\") pod \"maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:08.575382 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.575355    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:08.669081 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.669046    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:08.678040 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.678015    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:08.735894 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.735861    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"]
Jul 24 15:09:08.736414 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.736390    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy"
Jul 24 15:09:08.736414 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.736415    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy"
Jul 24 15:09:08.736623 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.736543    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2216c91f-57d8-48cb-a4e3-6bc0433702e4" containerName="istio-proxy"
Jul 24 15:09:08.740024 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.739996    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:08.747528 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.747234    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e61f3ca0\""
Jul 24 15:09:08.749113 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.748654    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e61f3ca0-dockercfg-2dl29\""
Jul 24 15:09:08.750868 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.750816    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"]
Jul 24 15:09:08.795996 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.795957    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"]
Jul 24 15:09:08.800004 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.799985    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:08.814018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.813593    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"]
Jul 24 15:09:08.816241 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.816223    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"]
Jul 24 15:09:08.818427 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:09:08.818400    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b380a7_3345_4742_a6ce_78591805e274.slice/crio-c2f605c221dfada1a725074a2969b82d922949dd6a3f09c89b6ed420a5148f22 WatchSource:0}: Error finding container c2f605c221dfada1a725074a2969b82d922949dd6a3f09c89b6ed420a5148f22: Status 404 returned error can't find the container with id c2f605c221dfada1a725074a2969b82d922949dd6a3f09c89b6ed420a5148f22
Jul 24 15:09:08.820408 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.819955    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:09:08.859349 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.859325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r\" (UID: \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:08.859490 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.859378    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj2x2\" (UniqueName: \"kubernetes.io/projected/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-kube-api-access-mj2x2\") pod \"payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r\" (UID: \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:08.960734 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.960702    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj2x2\" (UniqueName: \"kubernetes.io/projected/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-kube-api-access-mj2x2\") pod \"payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r\" (UID: \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:08.960843 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.960762    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzgsz\" (UniqueName: \"kubernetes.io/projected/9feff163-06bb-4f30-9796-200d4f7034d2-kube-api-access-wzgsz\") pod \"payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l\" (UID: \"9feff163-06bb-4f30-9796-200d4f7034d2\") " pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:08.960843 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.960802    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r\" (UID: \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:08.960843 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.960822    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9feff163-06bb-4f30-9796-200d4f7034d2-plugins-config-volume\") pod \"payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l\" (UID: \"9feff163-06bb-4f30-9796-200d4f7034d2\") " pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:08.961403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.961386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r\" (UID: \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:08.968297 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:08.968273    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj2x2\" (UniqueName: \"kubernetes.io/projected/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-kube-api-access-mj2x2\") pod \"payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r\" (UID: \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:09.053731 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.053677    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:09.061362 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.061338    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzgsz\" (UniqueName: \"kubernetes.io/projected/9feff163-06bb-4f30-9796-200d4f7034d2-kube-api-access-wzgsz\") pod \"payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l\" (UID: \"9feff163-06bb-4f30-9796-200d4f7034d2\") " pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:09.061497 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.061391    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9feff163-06bb-4f30-9796-200d4f7034d2-plugins-config-volume\") pod \"payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l\" (UID: \"9feff163-06bb-4f30-9796-200d4f7034d2\") " pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:09.061957 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.061937    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9feff163-06bb-4f30-9796-200d4f7034d2-plugins-config-volume\") pod \"payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l\" (UID: \"9feff163-06bb-4f30-9796-200d4f7034d2\") " pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:09.071547 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.071524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzgsz\" (UniqueName: \"kubernetes.io/projected/9feff163-06bb-4f30-9796-200d4f7034d2-kube-api-access-wzgsz\") pod \"payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l\" (UID: \"9feff163-06bb-4f30-9796-200d4f7034d2\") " pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:09.111378 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.111345    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:09.210105 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:09:09.210074    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd63e3dbf_ef5d_424a_aff2_c604b8cfd054.slice/crio-6af25554a7395d21f2e7fac43dc6dd33c97782f4eed207733056dc37eb437ee3 WatchSource:0}: Error finding container 6af25554a7395d21f2e7fac43dc6dd33c97782f4eed207733056dc37eb437ee3: Status 404 returned error can't find the container with id 6af25554a7395d21f2e7fac43dc6dd33c97782f4eed207733056dc37eb437ee3
Jul 24 15:09:09.212066 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.212042    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"]
Jul 24 15:09:09.266712 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.266663    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"]
Jul 24 15:09:09.268999 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:09:09.268975    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9feff163_06bb_4f30_9796_200d4f7034d2.slice/crio-043a5cb0e3c97d7fa8dd5ae1ae7069bd1b1f7a0f7c37b4ba06898ee8490c528b WatchSource:0}: Error finding container 043a5cb0e3c97d7fa8dd5ae1ae7069bd1b1f7a0f7c37b4ba06898ee8490c528b: Status 404 returned error can't find the container with id 043a5cb0e3c97d7fa8dd5ae1ae7069bd1b1f7a0f7c37b4ba06898ee8490c528b
Jul 24 15:09:09.511070 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.511019    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l" event={"ID":"9feff163-06bb-4f30-9796-200d4f7034d2","Type":"ContainerStarted","Data":"8b83059809038cadb9c371fe8b591e3a06f24b2fbead7db7ccf179d3f0073185"}
Jul 24 15:09:09.511247 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.511090    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l" event={"ID":"9feff163-06bb-4f30-9796-200d4f7034d2","Type":"ContainerStarted","Data":"043a5cb0e3c97d7fa8dd5ae1ae7069bd1b1f7a0f7c37b4ba06898ee8490c528b"}
Jul 24 15:09:09.511247 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.511125    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:09.512519 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.512493    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj" event={"ID":"18b380a7-3345-4742-a6ce-78591805e274","Type":"ContainerStarted","Data":"3ce753f2c95b45476c646830c4f1a18a83033f367f4622ff9d43f11a3af4efbf"}
Jul 24 15:09:09.512648 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.512535    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj" event={"ID":"18b380a7-3345-4742-a6ce-78591805e274","Type":"ContainerStarted","Data":"c2f605c221dfada1a725074a2969b82d922949dd6a3f09c89b6ed420a5148f22"}
Jul 24 15:09:09.512648 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.512561    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:09.513799 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.513772    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r" event={"ID":"d63e3dbf-ef5d-424a-aff2-c604b8cfd054","Type":"ContainerStarted","Data":"f536c37fd0f09c63d66cc457b89cd57ec246b5167ca0349b932d8462f74f1d61"}
Jul 24 15:09:09.513880 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.513813    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r" event={"ID":"d63e3dbf-ef5d-424a-aff2-c604b8cfd054","Type":"ContainerStarted","Data":"6af25554a7395d21f2e7fac43dc6dd33c97782f4eed207733056dc37eb437ee3"}
Jul 24 15:09:09.513880 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.513831    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:09.544600 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.544555    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l" podStartSLOduration=1.544538618 podStartE2EDuration="1.544538618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:09:09.541024713 +0000 UTC m=+2134.694025438" watchObservedRunningTime="2026-07-24 15:09:09.544538618 +0000 UTC m=+2134.697539346"
Jul 24 15:09:09.569327 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.569249    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r" podStartSLOduration=1.569231869 podStartE2EDuration="1.569231869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:09:09.567258379 +0000 UTC m=+2134.720259103" watchObservedRunningTime="2026-07-24 15:09:09.569231869 +0000 UTC m=+2134.722232595"
Jul 24 15:09:09.596640 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:09.596601    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj" podStartSLOduration=2.140475403 podStartE2EDuration="2.596588746s" totalImagesPullingTime="456.113343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:09:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:09:09.596144158 +0000 UTC m=+2134.749144882" watchObservedRunningTime="2026-07-24 15:09:09.596588746 +0000 UTC m=+2134.749589470"
Jul 24 15:09:11.861147 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:11.861117    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:15.523195 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:15.523168    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:09:20.519594 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:20.519562    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:09:20.520005 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:20.519886    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:09:42.602400 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:09:42.602367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:08.329020 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:10:08.328990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:25.844482 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:25.844437    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"]
Jul 24 15:11:25.848553 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:25.844690    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj" podUID="18b380a7-3345-4742-a6ce-78591805e274" containerName="maas-api" containerID="cri-o://3ce753f2c95b45476c646830c4f1a18a83033f367f4622ff9d43f11a3af4efbf" gracePeriod=30
Jul 24 15:11:25.876371 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:25.876339    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:25.933690 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:25.933649    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"]
Jul 24 15:11:25.933897 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:25.933872    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l" podUID="9feff163-06bb-4f30-9796-200d4f7034d2" containerName="payload-processing" containerID="cri-o://8b83059809038cadb9c371fe8b591e3a06f24b2fbead7db7ccf179d3f0073185" gracePeriod=30
Jul 24 15:11:25.946150 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:25.946125    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"]
Jul 24 15:11:25.946322 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:25.946299    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r" podUID="d63e3dbf-ef5d-424a-aff2-c604b8cfd054" containerName="payload-pre-processing" containerID="cri-o://f536c37fd0f09c63d66cc457b89cd57ec246b5167ca0349b932d8462f74f1d61" gracePeriod=30
Jul 24 15:11:25.994858 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:25.994822    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:26.117174 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.117137    2583 generic.go:356] "Generic (PLEG): container finished" podID="d63e3dbf-ef5d-424a-aff2-c604b8cfd054" containerID="f536c37fd0f09c63d66cc457b89cd57ec246b5167ca0349b932d8462f74f1d61" exitCode=0
Jul 24 15:11:26.117328 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.117214    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r" event={"ID":"d63e3dbf-ef5d-424a-aff2-c604b8cfd054","Type":"ContainerDied","Data":"f536c37fd0f09c63d66cc457b89cd57ec246b5167ca0349b932d8462f74f1d61"}
Jul 24 15:11:26.118915 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.118888    2583 generic.go:356] "Generic (PLEG): container finished" podID="9feff163-06bb-4f30-9796-200d4f7034d2" containerID="8b83059809038cadb9c371fe8b591e3a06f24b2fbead7db7ccf179d3f0073185" exitCode=0
Jul 24 15:11:26.119038 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.118955    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l" event={"ID":"9feff163-06bb-4f30-9796-200d4f7034d2","Type":"ContainerDied","Data":"8b83059809038cadb9c371fe8b591e3a06f24b2fbead7db7ccf179d3f0073185"}
Jul 24 15:11:26.120574 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.120553    2583 generic.go:356] "Generic (PLEG): container finished" podID="18b380a7-3345-4742-a6ce-78591805e274" containerID="3ce753f2c95b45476c646830c4f1a18a83033f367f4622ff9d43f11a3af4efbf" exitCode=0
Jul 24 15:11:26.120669 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.120578    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj" event={"ID":"18b380a7-3345-4742-a6ce-78591805e274","Type":"ContainerDied","Data":"3ce753f2c95b45476c646830c4f1a18a83033f367f4622ff9d43f11a3af4efbf"}
Jul 24 15:11:26.146771 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.146746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:26.193861 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.193828    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:11:26.225799 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.225772    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:11:26.234670 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.234645    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:11:26.242269 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.242245    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj2x2\" (UniqueName: \"kubernetes.io/projected/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-kube-api-access-mj2x2\") pod \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\" (UID: \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\") "
Jul 24 15:11:26.242403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.242291    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpccw\" (UniqueName: \"kubernetes.io/projected/18b380a7-3345-4742-a6ce-78591805e274-kube-api-access-zpccw\") pod \"18b380a7-3345-4742-a6ce-78591805e274\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") "
Jul 24 15:11:26.242403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.242333    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18b380a7-3345-4742-a6ce-78591805e274-maas-api-tls\") pod \"18b380a7-3345-4742-a6ce-78591805e274\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") "
Jul 24 15:11:26.242403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.242376    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b380a7-3345-4742-a6ce-78591805e274-openshift-service-ca\") pod \"18b380a7-3345-4742-a6ce-78591805e274\" (UID: \"18b380a7-3345-4742-a6ce-78591805e274\") "
Jul 24 15:11:26.242403 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.242401    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzgsz\" (UniqueName: \"kubernetes.io/projected/9feff163-06bb-4f30-9796-200d4f7034d2-kube-api-access-wzgsz\") pod \"9feff163-06bb-4f30-9796-200d4f7034d2\" (UID: \"9feff163-06bb-4f30-9796-200d4f7034d2\") "
Jul 24 15:11:26.242642 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.242428    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9feff163-06bb-4f30-9796-200d4f7034d2-plugins-config-volume\") pod \"9feff163-06bb-4f30-9796-200d4f7034d2\" (UID: \"9feff163-06bb-4f30-9796-200d4f7034d2\") "
Jul 24 15:11:26.242642 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.242535    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-plugins-config-volume\") pod \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\" (UID: \"d63e3dbf-ef5d-424a-aff2-c604b8cfd054\") "
Jul 24 15:11:26.243394 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.243135    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d63e3dbf-ef5d-424a-aff2-c604b8cfd054" (UID: "d63e3dbf-ef5d-424a-aff2-c604b8cfd054"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:11:26.243394 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.243337    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18b380a7-3345-4742-a6ce-78591805e274-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18b380a7-3345-4742-a6ce-78591805e274" (UID: "18b380a7-3345-4742-a6ce-78591805e274"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:11:26.243590 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.243449    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9feff163-06bb-4f30-9796-200d4f7034d2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9feff163-06bb-4f30-9796-200d4f7034d2" (UID: "9feff163-06bb-4f30-9796-200d4f7034d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:11:26.244716 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.244673    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-kube-api-access-mj2x2" (OuterVolumeSpecName: "kube-api-access-mj2x2") pod "d63e3dbf-ef5d-424a-aff2-c604b8cfd054" (UID: "d63e3dbf-ef5d-424a-aff2-c604b8cfd054"). InnerVolumeSpecName "kube-api-access-mj2x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:11:26.244925 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.244900    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18b380a7-3345-4742-a6ce-78591805e274-kube-api-access-zpccw" (OuterVolumeSpecName: "kube-api-access-zpccw") pod "18b380a7-3345-4742-a6ce-78591805e274" (UID: "18b380a7-3345-4742-a6ce-78591805e274"). InnerVolumeSpecName "kube-api-access-zpccw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:11:26.245304 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.245268    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9feff163-06bb-4f30-9796-200d4f7034d2-kube-api-access-wzgsz" (OuterVolumeSpecName: "kube-api-access-wzgsz") pod "9feff163-06bb-4f30-9796-200d4f7034d2" (UID: "9feff163-06bb-4f30-9796-200d4f7034d2"). InnerVolumeSpecName "kube-api-access-wzgsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:11:26.245675 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.245651    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18b380a7-3345-4742-a6ce-78591805e274-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18b380a7-3345-4742-a6ce-78591805e274" (UID: "18b380a7-3345-4742-a6ce-78591805e274"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:11:26.344185 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.344153    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:11:26.344185 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.344177    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj2x2\" (UniqueName: \"kubernetes.io/projected/d63e3dbf-ef5d-424a-aff2-c604b8cfd054-kube-api-access-mj2x2\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:11:26.344185 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.344187    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpccw\" (UniqueName: \"kubernetes.io/projected/18b380a7-3345-4742-a6ce-78591805e274-kube-api-access-zpccw\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:11:26.344494 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.344196    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18b380a7-3345-4742-a6ce-78591805e274-maas-api-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:11:26.344494 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.344209    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b380a7-3345-4742-a6ce-78591805e274-openshift-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:11:26.344494 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.344223    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzgsz\" (UniqueName: \"kubernetes.io/projected/9feff163-06bb-4f30-9796-200d4f7034d2-kube-api-access-wzgsz\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:11:26.344494 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:26.344234    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9feff163-06bb-4f30-9796-200d4f7034d2-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:11:27.126664 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.126590    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"
Jul 24 15:11:27.126664 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.126605    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r" event={"ID":"d63e3dbf-ef5d-424a-aff2-c604b8cfd054","Type":"ContainerDied","Data":"6af25554a7395d21f2e7fac43dc6dd33c97782f4eed207733056dc37eb437ee3"}
Jul 24 15:11:27.127122 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.126677    2583 scope.go:117] "RemoveContainer" containerID="f536c37fd0f09c63d66cc457b89cd57ec246b5167ca0349b932d8462f74f1d61"
Jul 24 15:11:27.128144 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.128114    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l" event={"ID":"9feff163-06bb-4f30-9796-200d4f7034d2","Type":"ContainerDied","Data":"043a5cb0e3c97d7fa8dd5ae1ae7069bd1b1f7a0f7c37b4ba06898ee8490c528b"}
Jul 24 15:11:27.128273 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.128165    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"
Jul 24 15:11:27.129524 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.129501    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj" event={"ID":"18b380a7-3345-4742-a6ce-78591805e274","Type":"ContainerDied","Data":"c2f605c221dfada1a725074a2969b82d922949dd6a3f09c89b6ed420a5148f22"}
Jul 24 15:11:27.138258 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.129666    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"
Jul 24 15:11:27.143736 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.143721    2583 scope.go:117] "RemoveContainer" containerID="8b83059809038cadb9c371fe8b591e3a06f24b2fbead7db7ccf179d3f0073185"
Jul 24 15:11:27.152199 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.152180    2583 scope.go:117] "RemoveContainer" containerID="3ce753f2c95b45476c646830c4f1a18a83033f367f4622ff9d43f11a3af4efbf"
Jul 24 15:11:27.158900 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.158838    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"]
Jul 24 15:11:27.160843 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.160824    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r"]
Jul 24 15:11:27.170498 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.170447    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"]
Jul 24 15:11:27.174547 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.174529    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e61f3ca0-77d9b5779c-xjwxj"]
Jul 24 15:11:27.184412 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.184393    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"]
Jul 24 15:11:27.188501 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.188482    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l"]
Jul 24 15:11:27.342123 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.342101    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18b380a7-3345-4742-a6ce-78591805e274" path="/var/lib/kubelet/pods/18b380a7-3345-4742-a6ce-78591805e274/volumes"
Jul 24 15:11:27.342434 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.342421    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9feff163-06bb-4f30-9796-200d4f7034d2" path="/var/lib/kubelet/pods/9feff163-06bb-4f30-9796-200d4f7034d2/volumes"
Jul 24 15:11:27.342734 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:27.342722    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d63e3dbf-ef5d-424a-aff2-c604b8cfd054" path="/var/lib/kubelet/pods/d63e3dbf-ef5d-424a-aff2-c604b8cfd054/volumes"
Jul 24 15:11:30.405820 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:30.405787    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:30.897273 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:30.897242    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:35.087723 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:35.087688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:35.188912 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:35.188881    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:37.120946 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:37.120910    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:37.220518 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:37.220487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:37.392334 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:37.392244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:37.620643 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:37.620612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:38.812834 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:38.812802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:38.912256 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:38.912227    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:39.035691 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:39.035662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:39.873880 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:39.873848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:39.966867 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:39.966842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:40.069390 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:40.069362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:41.576197 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:41.576168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:48.699747 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:48.699714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:48.796607 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:48.796576    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:49.624559 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:49.624518    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:56.467340 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:56.467306    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:56.569738 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:56.569708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:56.677863 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:11:56.677836    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:07.237310 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:07.237274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:07.552869 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:07.552792    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:28.603684 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:28.603603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:35.908872 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:35.908841    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:36.012014 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.011982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:36.274804 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.274739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:36.784828 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.784797    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"]
Jul 24 15:12:36.785138 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.785125    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9feff163-06bb-4f30-9796-200d4f7034d2" containerName="payload-processing"
Jul 24 15:12:36.785187 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.785139    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9feff163-06bb-4f30-9796-200d4f7034d2" containerName="payload-processing"
Jul 24 15:12:36.785187 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.785149    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18b380a7-3345-4742-a6ce-78591805e274" containerName="maas-api"
Jul 24 15:12:36.785187 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.785154    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="18b380a7-3345-4742-a6ce-78591805e274" containerName="maas-api"
Jul 24 15:12:36.785187 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.785168    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d63e3dbf-ef5d-424a-aff2-c604b8cfd054" containerName="payload-pre-processing"
Jul 24 15:12:36.785187 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.785174    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d63e3dbf-ef5d-424a-aff2-c604b8cfd054" containerName="payload-pre-processing"
Jul 24 15:12:36.785333 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.785232    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d63e3dbf-ef5d-424a-aff2-c604b8cfd054" containerName="payload-pre-processing"
Jul 24 15:12:36.785333 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.785242    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="18b380a7-3345-4742-a6ce-78591805e274" containerName="maas-api"
Jul 24 15:12:36.785333 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.785248    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9feff163-06bb-4f30-9796-200d4f7034d2" containerName="payload-processing"
Jul 24 15:12:36.788130 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.788109    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:36.791765 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.791742    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6c969d-openshift-default-dockercfg-99f5q\""
Jul 24 15:12:36.810412 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.810392    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"]
Jul 24 15:12:36.931238 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.931197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:36.964508 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.964448    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-credential-socket\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:36.964508 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.964512    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-envoy\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:36.964672 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.964573    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-podinfo\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:36.964672 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.964608    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-token\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:36.964672 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.964637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-socket\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:36.964779 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.964697    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-certs\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:36.964779 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.964722    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5ztn\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-kube-api-access-m5ztn\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:36.964779 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.964750    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-data\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:36.964779 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:36.964769    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ceeb8d-8db0-4320-a186-4dd735a0002e-istiod-ca-cert\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066079 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066022    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-credential-socket\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066079 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-envoy\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066079 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066075    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-podinfo\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066267 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-token\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066267 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-socket\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066267 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066186    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-certs\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066267 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066213    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5ztn\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-kube-api-access-m5ztn\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066267 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066237    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-data\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066267 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066262    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ceeb8d-8db0-4320-a186-4dd735a0002e-istiod-ca-cert\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066595 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066404    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-credential-socket\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066595 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066502    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-certs\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066693 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066633    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-socket\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066693 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066660    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-data\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.066960 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.066941    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ceeb8d-8db0-4320-a186-4dd735a0002e-istiod-ca-cert\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.068218 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.068199    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-envoy\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.068429 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.068412    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-podinfo\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.075328 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.075307    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-token\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.075404 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.075358    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5ztn\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-kube-api-access-m5ztn\") pod \"e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") " pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.098335 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.098314    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:37.232505 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.232393    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"]
Jul 24 15:12:37.235013 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:12:37.234946    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ceeb8d_8db0_4320_a186_4dd735a0002e.slice/crio-25f13b056eeab9211e8194d4408ecb684a198ebb6a023e0793862717311a2968 WatchSource:0}: Error finding container 25f13b056eeab9211e8194d4408ecb684a198ebb6a023e0793862717311a2968: Status 404 returned error can't find the container with id 25f13b056eeab9211e8194d4408ecb684a198ebb6a023e0793862717311a2968
Jul 24 15:12:37.237502 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.237443    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:12:37.237610 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.237555    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:12:37.237610 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.237599    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:12:37.450204 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.450171    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" event={"ID":"93ceeb8d-8db0-4320-a186-4dd735a0002e","Type":"ContainerStarted","Data":"e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc"}
Jul 24 15:12:37.450392 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.450224    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" event={"ID":"93ceeb8d-8db0-4320-a186-4dd735a0002e","Type":"ContainerStarted","Data":"25f13b056eeab9211e8194d4408ecb684a198ebb6a023e0793862717311a2968"}
Jul 24 15:12:37.471946 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.471900    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" podStartSLOduration=1.471882443 podStartE2EDuration="1.471882443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:12:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:12:37.471138682 +0000 UTC m=+2342.624139407" watchObservedRunningTime="2026-07-24 15:12:37.471882443 +0000 UTC m=+2342.624883169"
Jul 24 15:12:37.474400 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:37.474380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:38.099021 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:38.098991    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:38.103613 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:38.103589    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:38.456407 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:38.456319    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:38.457279 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:38.457258    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:39.504629 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.504588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:39.626648 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.626618    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"]
Jul 24 15:12:39.628571 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.628546    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:39.629758 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.629740    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:39.632544 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.632524    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6c969d-serving-cert\""
Jul 24 15:12:39.649889 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.649869    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"]
Jul 24 15:12:39.721729 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.721701    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:39.789046 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.788981    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071fff2c-2922-4688-bd3d-c45a286276b5-maas-api-tls\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:39.789046 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.789027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071fff2c-2922-4688-bd3d-c45a286276b5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:39.789229 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.789137    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqprx\" (UniqueName: \"kubernetes.io/projected/071fff2c-2922-4688-bd3d-c45a286276b5-kube-api-access-kqprx\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:39.889985 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.889958    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqprx\" (UniqueName: \"kubernetes.io/projected/071fff2c-2922-4688-bd3d-c45a286276b5-kube-api-access-kqprx\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:39.890087 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.890005    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071fff2c-2922-4688-bd3d-c45a286276b5-maas-api-tls\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:39.890087 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.890053    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071fff2c-2922-4688-bd3d-c45a286276b5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:39.890164 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:12:39.890105    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-serving-cert: secret "maas-api-e2e-inf-a-6c969d-serving-cert" not found
Jul 24 15:12:39.890197 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:12:39.890166    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/071fff2c-2922-4688-bd3d-c45a286276b5-maas-api-tls podName:071fff2c-2922-4688-bd3d-c45a286276b5 nodeName:}" failed. No retries permitted until 2026-07-24 15:12:40.390150363 +0000 UTC m=+2345.543151067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/071fff2c-2922-4688-bd3d-c45a286276b5-maas-api-tls") pod "maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd" (UID: "071fff2c-2922-4688-bd3d-c45a286276b5") : secret "maas-api-e2e-inf-a-6c969d-serving-cert" not found
Jul 24 15:12:39.890698 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.890679    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071fff2c-2922-4688-bd3d-c45a286276b5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:39.902675 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:39.902656    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqprx\" (UniqueName: \"kubernetes.io/projected/071fff2c-2922-4688-bd3d-c45a286276b5-kube-api-access-kqprx\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:40.192920 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.192887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:40.294557 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.294520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:40.327133 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.327107    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"]
Jul 24 15:12:40.330623 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.330602    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:40.333396 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.333378    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-6c969d-dockercfg-v94v5\""
Jul 24 15:12:40.333912 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.333895    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-6c969d\""
Jul 24 15:12:40.342481 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.342439    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"]
Jul 24 15:12:40.347952 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.347932    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"]
Jul 24 15:12:40.351026 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.351011    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:40.366320 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.366298    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"]
Jul 24 15:12:40.394351 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.394329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071fff2c-2922-4688-bd3d-c45a286276b5-maas-api-tls\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:40.396605 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.396584    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071fff2c-2922-4688-bd3d-c45a286276b5-maas-api-tls\") pod \"maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:40.398962 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.398941    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:40.495649 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.495567    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdfmp\" (UniqueName: \"kubernetes.io/projected/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-kube-api-access-sdfmp\") pod \"payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n\" (UID: \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:40.495649 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.495629    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jg9b\" (UniqueName: \"kubernetes.io/projected/543143f7-190b-46fc-a851-45074397d7af-kube-api-access-5jg9b\") pod \"payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb\" (UID: \"543143f7-190b-46fc-a851-45074397d7af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:40.495824 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.495666    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543143f7-190b-46fc-a851-45074397d7af-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb\" (UID: \"543143f7-190b-46fc-a851-45074397d7af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:40.495824 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.495715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n\" (UID: \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:40.540576 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.540552    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:40.596920 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.596889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543143f7-190b-46fc-a851-45074397d7af-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb\" (UID: \"543143f7-190b-46fc-a851-45074397d7af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:40.597057 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.596960    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n\" (UID: \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:40.597118 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.597078    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdfmp\" (UniqueName: \"kubernetes.io/projected/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-kube-api-access-sdfmp\") pod \"payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n\" (UID: \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:40.597171 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.597121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jg9b\" (UniqueName: \"kubernetes.io/projected/543143f7-190b-46fc-a851-45074397d7af-kube-api-access-5jg9b\") pod \"payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb\" (UID: \"543143f7-190b-46fc-a851-45074397d7af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:40.597818 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.597789    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543143f7-190b-46fc-a851-45074397d7af-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb\" (UID: \"543143f7-190b-46fc-a851-45074397d7af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:40.598155 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.598129    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n\" (UID: \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:40.607851 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.607806    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jg9b\" (UniqueName: \"kubernetes.io/projected/543143f7-190b-46fc-a851-45074397d7af-kube-api-access-5jg9b\") pod \"payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb\" (UID: \"543143f7-190b-46fc-a851-45074397d7af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:40.608984 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.608962    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdfmp\" (UniqueName: \"kubernetes.io/projected/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-kube-api-access-sdfmp\") pod \"payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n\" (UID: \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:40.649472 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.645940    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:40.661828 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.661770    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:40.698578 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.698386    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"]
Jul 24 15:12:40.701978 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:12:40.701944    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071fff2c_2922_4688_bd3d_c45a286276b5.slice/crio-7dd7f6171e38d966e41f2829d29d9412034df734b5349dcc59dd9612bd26bb83 WatchSource:0}: Error finding container 7dd7f6171e38d966e41f2829d29d9412034df734b5349dcc59dd9612bd26bb83: Status 404 returned error can't find the container with id 7dd7f6171e38d966e41f2829d29d9412034df734b5349dcc59dd9612bd26bb83
Jul 24 15:12:40.808676 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:40.808491    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"]
Jul 24 15:12:41.036625 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.036597    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"]
Jul 24 15:12:41.038705 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:12:41.038654    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod543143f7_190b_46fc_a851_45074397d7af.slice/crio-bab357dccd6391520eef11d24d615ab1c03463a1450fee98ae95bc8adb3e77b2 WatchSource:0}: Error finding container bab357dccd6391520eef11d24d615ab1c03463a1450fee98ae95bc8adb3e77b2: Status 404 returned error can't find the container with id bab357dccd6391520eef11d24d615ab1c03463a1450fee98ae95bc8adb3e77b2
Jul 24 15:12:41.471758 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.471720    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n" event={"ID":"bc4b9bce-5398-41b8-9416-62d9ce2a1e18","Type":"ContainerStarted","Data":"3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00"}
Jul 24 15:12:41.471924 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.471779    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n" event={"ID":"bc4b9bce-5398-41b8-9416-62d9ce2a1e18","Type":"ContainerStarted","Data":"e69a41881a688d991641e1b3ae847b753c1c5dc74a576abfb40a86727c7765a6"}
Jul 24 15:12:41.471994 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.471908    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:41.473360 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.473333    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd" event={"ID":"071fff2c-2922-4688-bd3d-c45a286276b5","Type":"ContainerStarted","Data":"242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e"}
Jul 24 15:12:41.473499 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.473380    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd" event={"ID":"071fff2c-2922-4688-bd3d-c45a286276b5","Type":"ContainerStarted","Data":"7dd7f6171e38d966e41f2829d29d9412034df734b5349dcc59dd9612bd26bb83"}
Jul 24 15:12:41.473499 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.473407    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:41.474864 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.474843    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb" event={"ID":"543143f7-190b-46fc-a851-45074397d7af","Type":"ContainerStarted","Data":"52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b"}
Jul 24 15:12:41.474943 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.474891    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb" event={"ID":"543143f7-190b-46fc-a851-45074397d7af","Type":"ContainerStarted","Data":"bab357dccd6391520eef11d24d615ab1c03463a1450fee98ae95bc8adb3e77b2"}
Jul 24 15:12:41.474943 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.474917    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:41.519671 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.519633    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n" podStartSLOduration=1.519621986 podStartE2EDuration="1.519621986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:12:41.496632616 +0000 UTC m=+2346.649633341" watchObservedRunningTime="2026-07-24 15:12:41.519621986 +0000 UTC m=+2346.672622711"
Jul 24 15:12:41.520068 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.520039    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd" podStartSLOduration=2.046149739 podStartE2EDuration="2.520030676s" totalImagesPullingTime="473.880937ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:12:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:12:41.517980179 +0000 UTC m=+2346.670980904" watchObservedRunningTime="2026-07-24 15:12:41.520030676 +0000 UTC m=+2346.673031445"
Jul 24 15:12:41.538994 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:41.538953    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb" podStartSLOduration=1.5389403499999998 podStartE2EDuration="1.53894035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:12:41.537102694 +0000 UTC m=+2346.690103418" watchObservedRunningTime="2026-07-24 15:12:41.53894035 +0000 UTC m=+2346.691941077"
Jul 24 15:12:44.128620 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:44.128577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:47.485482 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:47.485435    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:12:50.521633 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:50.521581    2583 generic.go:356] "Generic (PLEG): container finished" podID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerID="e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc" exitCode=0
Jul 24 15:12:50.521633 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:50.521615    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" event={"ID":"93ceeb8d-8db0-4320-a186-4dd735a0002e","Type":"ContainerDied","Data":"e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc"}
Jul 24 15:12:50.521964 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:50.521910    2583 scope.go:117] "RemoveContainer" containerID="e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc"
Jul 24 15:12:50.522830 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:50.522800    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:12:50.522914 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:50.522865    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:12:50.522914 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:50.522896    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 24 15:12:51.527977 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:51.527946    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" event={"ID":"93ceeb8d-8db0-4320-a186-4dd735a0002e","Type":"ContainerStarted","Data":"16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f"}
Jul 24 15:12:52.099354 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:52.099319    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:52.100286 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:52.100259    2583 patch_prober.go:28] interesting pod/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 24 15:12:52.100380 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:52.100317    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 24 15:12:52.481641 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:52.481556    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:12:52.481641 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:52.481634    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:12:53.098908 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:53.098873    2583 patch_prober.go:28] interesting pod/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 24 15:12:53.099282 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:53.098935    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 24 15:12:53.456609 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:53.456512    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:54.099588 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:54.099550    2583 patch_prober.go:28] interesting pod/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 24 15:12:54.099941 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:54.099608    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 24 15:12:55.103070 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:55.103034    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:12:55.103995 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:12:55.103976    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:14:02.987219 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:02.987149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:14:58.647542 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:58.647507    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"]
Jul 24 15:14:58.648022 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:58.647719    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd" podUID="071fff2c-2922-4688-bd3d-c45a286276b5" containerName="maas-api" containerID="cri-o://242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e" gracePeriod=30
Jul 24 15:14:58.718326 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:58.718297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:14:58.765634 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:58.765608    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"]
Jul 24 15:14:58.765853 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:58.765827    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb" podUID="543143f7-190b-46fc-a851-45074397d7af" containerName="payload-processing" containerID="cri-o://52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b" gracePeriod=30
Jul 24 15:14:58.778197 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:58.778171    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"]
Jul 24 15:14:58.778404 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:58.778380    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n" podUID="bc4b9bce-5398-41b8-9416-62d9ce2a1e18" containerName="payload-pre-processing" containerID="cri-o://3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00" gracePeriod=30
Jul 24 15:14:58.912184 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:58.912107    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:14:59.027042 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.027010    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:14:59.034197 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.034176    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071fff2c-2922-4688-bd3d-c45a286276b5-maas-api-tls\") pod \"071fff2c-2922-4688-bd3d-c45a286276b5\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") "
Jul 24 15:14:59.034332 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.034216    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqprx\" (UniqueName: \"kubernetes.io/projected/071fff2c-2922-4688-bd3d-c45a286276b5-kube-api-access-kqprx\") pod \"071fff2c-2922-4688-bd3d-c45a286276b5\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") "
Jul 24 15:14:59.034332 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.034252    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071fff2c-2922-4688-bd3d-c45a286276b5-openshift-service-ca\") pod \"071fff2c-2922-4688-bd3d-c45a286276b5\" (UID: \"071fff2c-2922-4688-bd3d-c45a286276b5\") "
Jul 24 15:14:59.034707 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.034664    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/071fff2c-2922-4688-bd3d-c45a286276b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "071fff2c-2922-4688-bd3d-c45a286276b5" (UID: "071fff2c-2922-4688-bd3d-c45a286276b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:14:59.035178 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.035156    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:14:59.036212 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.036190    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/071fff2c-2922-4688-bd3d-c45a286276b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "071fff2c-2922-4688-bd3d-c45a286276b5" (UID: "071fff2c-2922-4688-bd3d-c45a286276b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:14:59.036631 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.036611    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071fff2c-2922-4688-bd3d-c45a286276b5-kube-api-access-kqprx" (OuterVolumeSpecName: "kube-api-access-kqprx") pod "071fff2c-2922-4688-bd3d-c45a286276b5" (UID: "071fff2c-2922-4688-bd3d-c45a286276b5"). InnerVolumeSpecName "kube-api-access-kqprx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:14:59.100121 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.100082    2583 generic.go:356] "Generic (PLEG): container finished" podID="543143f7-190b-46fc-a851-45074397d7af" containerID="52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b" exitCode=0
Jul 24 15:14:59.100310 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.100130    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"
Jul 24 15:14:59.100310 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.100166    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb" event={"ID":"543143f7-190b-46fc-a851-45074397d7af","Type":"ContainerDied","Data":"52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b"}
Jul 24 15:14:59.100310 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.100255    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb" event={"ID":"543143f7-190b-46fc-a851-45074397d7af","Type":"ContainerDied","Data":"bab357dccd6391520eef11d24d615ab1c03463a1450fee98ae95bc8adb3e77b2"}
Jul 24 15:14:59.100310 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.100286    2583 scope.go:117] "RemoveContainer" containerID="52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b"
Jul 24 15:14:59.101444 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.101424    2583 generic.go:356] "Generic (PLEG): container finished" podID="bc4b9bce-5398-41b8-9416-62d9ce2a1e18" containerID="3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00" exitCode=0
Jul 24 15:14:59.101549 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.101485    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"
Jul 24 15:14:59.101549 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.101514    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n" event={"ID":"bc4b9bce-5398-41b8-9416-62d9ce2a1e18","Type":"ContainerDied","Data":"3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00"}
Jul 24 15:14:59.101667 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.101567    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n" event={"ID":"bc4b9bce-5398-41b8-9416-62d9ce2a1e18","Type":"ContainerDied","Data":"e69a41881a688d991641e1b3ae847b753c1c5dc74a576abfb40a86727c7765a6"}
Jul 24 15:14:59.102869 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.102850    2583 generic.go:356] "Generic (PLEG): container finished" podID="071fff2c-2922-4688-bd3d-c45a286276b5" containerID="242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e" exitCode=0
Jul 24 15:14:59.102970 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.102884    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd" event={"ID":"071fff2c-2922-4688-bd3d-c45a286276b5","Type":"ContainerDied","Data":"242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e"}
Jul 24 15:14:59.102970 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.102895    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"
Jul 24 15:14:59.102970 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.102918    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd" event={"ID":"071fff2c-2922-4688-bd3d-c45a286276b5","Type":"ContainerDied","Data":"7dd7f6171e38d966e41f2829d29d9412034df734b5349dcc59dd9612bd26bb83"}
Jul 24 15:14:59.109965 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.109947    2583 scope.go:117] "RemoveContainer" containerID="52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b"
Jul 24 15:14:59.110323 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:14:59.110279    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b\": container with ID starting with 52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b not found: ID does not exist" containerID="52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b"
Jul 24 15:14:59.110323 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.110305    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b"} err="failed to get container status \"52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b\": rpc error: code = NotFound desc = could not find container \"52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b\": container with ID starting with 52270790e9bc845ec7ea70a0a12b4ad27c8383d8f2846f154cdddd5507b30e4b not found: ID does not exist"
Jul 24 15:14:59.110480 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.110326    2583 scope.go:117] "RemoveContainer" containerID="3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00"
Jul 24 15:14:59.119290 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.119266    2583 scope.go:117] "RemoveContainer" containerID="3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00"
Jul 24 15:14:59.119659 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:14:59.119636    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00\": container with ID starting with 3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00 not found: ID does not exist" containerID="3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00"
Jul 24 15:14:59.119754 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.119669    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00"} err="failed to get container status \"3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00\": rpc error: code = NotFound desc = could not find container \"3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00\": container with ID starting with 3825d5bd52acdf5f673ab78b2c39a3fe30e1f90402ccf0d459a1efa9efa47b00 not found: ID does not exist"
Jul 24 15:14:59.119754 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.119688    2583 scope.go:117] "RemoveContainer" containerID="242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e"
Jul 24 15:14:59.127681 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.127664    2583 scope.go:117] "RemoveContainer" containerID="242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e"
Jul 24 15:14:59.127905 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:14:59.127885    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e\": container with ID starting with 242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e not found: ID does not exist" containerID="242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e"
Jul 24 15:14:59.127963 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.127913    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e"} err="failed to get container status \"242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e\": rpc error: code = NotFound desc = could not find container \"242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e\": container with ID starting with 242bffa2556e9a2302560ec3ada03c08d80d69bc9835b1f339c44e17fede654e not found: ID does not exist"
Jul 24 15:14:59.135298 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.135275    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-plugins-config-volume\") pod \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\" (UID: \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\") "
Jul 24 15:14:59.135363 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.135317    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdfmp\" (UniqueName: \"kubernetes.io/projected/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-kube-api-access-sdfmp\") pod \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\" (UID: \"bc4b9bce-5398-41b8-9416-62d9ce2a1e18\") "
Jul 24 15:14:59.135401 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.135385    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jg9b\" (UniqueName: \"kubernetes.io/projected/543143f7-190b-46fc-a851-45074397d7af-kube-api-access-5jg9b\") pod \"543143f7-190b-46fc-a851-45074397d7af\" (UID: \"543143f7-190b-46fc-a851-45074397d7af\") "
Jul 24 15:14:59.135446 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.135432    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543143f7-190b-46fc-a851-45074397d7af-plugins-config-volume\") pod \"543143f7-190b-46fc-a851-45074397d7af\" (UID: \"543143f7-190b-46fc-a851-45074397d7af\") "
Jul 24 15:14:59.135673 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.135648    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc4b9bce-5398-41b8-9416-62d9ce2a1e18" (UID: "bc4b9bce-5398-41b8-9416-62d9ce2a1e18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:14:59.135757 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.135709    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071fff2c-2922-4688-bd3d-c45a286276b5-maas-api-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:14:59.135757 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.135727    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqprx\" (UniqueName: \"kubernetes.io/projected/071fff2c-2922-4688-bd3d-c45a286276b5-kube-api-access-kqprx\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:14:59.135757 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.135742    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071fff2c-2922-4688-bd3d-c45a286276b5-openshift-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:14:59.135905 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.135884    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/543143f7-190b-46fc-a851-45074397d7af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "543143f7-190b-46fc-a851-45074397d7af" (UID: "543143f7-190b-46fc-a851-45074397d7af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:14:59.137272 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.137231    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/543143f7-190b-46fc-a851-45074397d7af-kube-api-access-5jg9b" (OuterVolumeSpecName: "kube-api-access-5jg9b") pod "543143f7-190b-46fc-a851-45074397d7af" (UID: "543143f7-190b-46fc-a851-45074397d7af"). InnerVolumeSpecName "kube-api-access-5jg9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:14:59.137341 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.137329    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-kube-api-access-sdfmp" (OuterVolumeSpecName: "kube-api-access-sdfmp") pod "bc4b9bce-5398-41b8-9416-62d9ce2a1e18" (UID: "bc4b9bce-5398-41b8-9416-62d9ce2a1e18"). InnerVolumeSpecName "kube-api-access-sdfmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:14:59.157560 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.157537    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"]
Jul 24 15:14:59.178722 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.178673    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c969d-55ccc96959-2t8xd"]
Jul 24 15:14:59.237101 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.237079    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:14:59.237188 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.237101    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdfmp\" (UniqueName: \"kubernetes.io/projected/bc4b9bce-5398-41b8-9416-62d9ce2a1e18-kube-api-access-sdfmp\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:14:59.237188 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.237116    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jg9b\" (UniqueName: \"kubernetes.io/projected/543143f7-190b-46fc-a851-45074397d7af-kube-api-access-5jg9b\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:14:59.237188 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.237127    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543143f7-190b-46fc-a851-45074397d7af-plugins-config-volume\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:14:59.342850 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.342826    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="071fff2c-2922-4688-bd3d-c45a286276b5" path="/var/lib/kubelet/pods/071fff2c-2922-4688-bd3d-c45a286276b5/volumes"
Jul 24 15:14:59.424762 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.424739    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"]
Jul 24 15:14:59.433291 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.433241    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb"]
Jul 24 15:14:59.457582 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.457560    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"]
Jul 24 15:14:59.467260 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:14:59.467242    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n"]
Jul 24 15:15:00.317665 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:00.317633    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:01.343102 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:01.343069    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="543143f7-190b-46fc-a851-45074397d7af" path="/var/lib/kubelet/pods/543143f7-190b-46fc-a851-45074397d7af/volumes"
Jul 24 15:15:01.343446 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:01.343412    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc4b9bce-5398-41b8-9416-62d9ce2a1e18" path="/var/lib/kubelet/pods/bc4b9bce-5398-41b8-9416-62d9ce2a1e18/volumes"
Jul 24 15:15:02.814864 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:02.814834    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:03.237761 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:03.237728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:09.583448 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:09.583388    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:09.685783 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:09.685753    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:09.902566 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:09.902535    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"]
Jul 24 15:15:09.902751 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:09.902730    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy" containerID="cri-o://16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f" gracePeriod=30
Jul 24 15:15:09.922987 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:09.922966    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:10.105844 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:10.105805    2583 patch_prober.go:28] interesting pod/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 15:15:10.105999 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:10.105877    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 15:15:10.151736 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:10.151709    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:11.654581 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:11.654550    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5846844f9f-jpfh2_62b14cb5-1877-4259-9464-db6f58c756d8/authorino/0.log"
Jul 24 15:15:15.148636 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.148612    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:15:15.181923 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.181867    2583 generic.go:356] "Generic (PLEG): container finished" podID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerID="16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f" exitCode=0
Jul 24 15:15:15.181923 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.181906    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"
Jul 24 15:15:15.182053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.181945    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" event={"ID":"93ceeb8d-8db0-4320-a186-4dd735a0002e","Type":"ContainerDied","Data":"16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f"}
Jul 24 15:15:15.182053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.182004    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz" event={"ID":"93ceeb8d-8db0-4320-a186-4dd735a0002e","Type":"ContainerDied","Data":"25f13b056eeab9211e8194d4408ecb684a198ebb6a023e0793862717311a2968"}
Jul 24 15:15:15.182053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.182028    2583 scope.go:117] "RemoveContainer" containerID="16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f"
Jul 24 15:15:15.189914 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.189896    2583 scope.go:117] "RemoveContainer" containerID="e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc"
Jul 24 15:15:15.197143 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.197128    2583 scope.go:117] "RemoveContainer" containerID="16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f"
Jul 24 15:15:15.197378 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:15:15.197359    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f\": container with ID starting with 16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f not found: ID does not exist" containerID="16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f"
Jul 24 15:15:15.197414 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.197385    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f"} err="failed to get container status \"16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f\": rpc error: code = NotFound desc = could not find container \"16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f\": container with ID starting with 16e5b6eb2e16ba7061c9b55d5a01cb45049566b58a51c812c8c0e3a65ad38c1f not found: ID does not exist"
Jul 24 15:15:15.197414 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.197401    2583 scope.go:117] "RemoveContainer" containerID="e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc"
Jul 24 15:15:15.197634 ip-10-0-141-133 kubenswrapper[2583]: E0724 15:15:15.197620    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc\": container with ID starting with e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc not found: ID does not exist" containerID="e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc"
Jul 24 15:15:15.197678 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.197639    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc"} err="failed to get container status \"e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc\": rpc error: code = NotFound desc = could not find container \"e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc\": container with ID starting with e36b8d1334ae37656006d7cb0354658f48d392e23c5600955b48e80cc3c73fcc not found: ID does not exist"
Jul 24 15:15:15.258827 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.258805    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-token\") pod \"93ceeb8d-8db0-4320-a186-4dd735a0002e\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") "
Jul 24 15:15:15.258903 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.258842    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-envoy\") pod \"93ceeb8d-8db0-4320-a186-4dd735a0002e\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") "
Jul 24 15:15:15.258903 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.258875    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-certs\") pod \"93ceeb8d-8db0-4320-a186-4dd735a0002e\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") "
Jul 24 15:15:15.258903 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.258899    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5ztn\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-kube-api-access-m5ztn\") pod \"93ceeb8d-8db0-4320-a186-4dd735a0002e\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") "
Jul 24 15:15:15.259008 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.258992    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-credential-socket\") pod \"93ceeb8d-8db0-4320-a186-4dd735a0002e\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") "
Jul 24 15:15:15.259046 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.259019    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-podinfo\") pod \"93ceeb8d-8db0-4320-a186-4dd735a0002e\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") "
Jul 24 15:15:15.259046 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.259041    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-data\") pod \"93ceeb8d-8db0-4320-a186-4dd735a0002e\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") "
Jul 24 15:15:15.259144 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.259080    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ceeb8d-8db0-4320-a186-4dd735a0002e-istiod-ca-cert\") pod \"93ceeb8d-8db0-4320-a186-4dd735a0002e\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") "
Jul 24 15:15:15.259144 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.259111    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-socket\") pod \"93ceeb8d-8db0-4320-a186-4dd735a0002e\" (UID: \"93ceeb8d-8db0-4320-a186-4dd735a0002e\") "
Jul 24 15:15:15.259244 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.259183    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93ceeb8d-8db0-4320-a186-4dd735a0002e" (UID: "93ceeb8d-8db0-4320-a186-4dd735a0002e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:15.259475 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.259321    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93ceeb8d-8db0-4320-a186-4dd735a0002e" (UID: "93ceeb8d-8db0-4320-a186-4dd735a0002e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:15.259475 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.259373    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:15:15.259616 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.259479    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ceeb8d-8db0-4320-a186-4dd735a0002e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93ceeb8d-8db0-4320-a186-4dd735a0002e" (UID: "93ceeb8d-8db0-4320-a186-4dd735a0002e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:15:15.259616 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.259487    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93ceeb8d-8db0-4320-a186-4dd735a0002e" (UID: "93ceeb8d-8db0-4320-a186-4dd735a0002e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:15.260050 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.260021    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-data" (OuterVolumeSpecName: "istio-data") pod "93ceeb8d-8db0-4320-a186-4dd735a0002e" (UID: "93ceeb8d-8db0-4320-a186-4dd735a0002e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:15.260777 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.260753    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93ceeb8d-8db0-4320-a186-4dd735a0002e" (UID: "93ceeb8d-8db0-4320-a186-4dd735a0002e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:15.261078 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.261052    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93ceeb8d-8db0-4320-a186-4dd735a0002e" (UID: "93ceeb8d-8db0-4320-a186-4dd735a0002e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:15:15.261137 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.261076    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-token" (OuterVolumeSpecName: "istio-token") pod "93ceeb8d-8db0-4320-a186-4dd735a0002e" (UID: "93ceeb8d-8db0-4320-a186-4dd735a0002e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:15:15.261137 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.261124    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-kube-api-access-m5ztn" (OuterVolumeSpecName: "kube-api-access-m5ztn") pod "93ceeb8d-8db0-4320-a186-4dd735a0002e" (UID: "93ceeb8d-8db0-4320-a186-4dd735a0002e"). InnerVolumeSpecName "kube-api-access-m5ztn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:15:15.359835 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.359815    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:15:15.359835 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.359835    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5ztn\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-kube-api-access-m5ztn\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:15:15.359937 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.359845    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:15:15.359937 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.359853    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:15:15.359937 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.359860    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:15:15.359937 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.359867    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ceeb8d-8db0-4320-a186-4dd735a0002e-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:15:15.359937 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.359874    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ceeb8d-8db0-4320-a186-4dd735a0002e-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:15:15.359937 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.359881    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ceeb8d-8db0-4320-a186-4dd735a0002e-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\""
Jul 24 15:15:15.498360 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.498312    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"]
Jul 24 15:15:15.501990 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:15.501967    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz"]
Jul 24 15:15:17.252142 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:17.252115    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c2lfm_43fde56f-fe9b-45a9-a5c5-7f7154cb6183/manager/0.log"
Jul 24 15:15:17.342410 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:17.342378    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" path="/var/lib/kubelet/pods/93ceeb8d-8db0-4320-a186-4dd735a0002e/volumes"
Jul 24 15:15:17.373939 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:17.373917    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kr4wf_11622fda-60f5-43aa-b613-ce594d477479/manager/0.log"
Jul 24 15:15:17.495837 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:17.495802    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7955497cfb-b4g79_07e49534-6a68-4595-a3ac-ba93fbfd98e6/manager/0.log"
Jul 24 15:15:17.627335 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:17.627321    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hlsvj_136fc1c8-5b98-4af4-8322-b7fbb686bb39/server/0.log"
Jul 24 15:15:17.751280 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:17.751257    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2dtr8_871167d0-5d94-47d5-8711-52bffbc5c9ae/manager/0.log"
Jul 24 15:15:17.877018 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:17.877001    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-696d5956bc-9m6l4_bc865c7d-dfe1-426b-91ec-f833e581e15c/copy-manifests/0.log"
Jul 24 15:15:17.890865 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:17.890804    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-696d5956bc-9m6l4_bc865c7d-dfe1-426b-91ec-f833e581e15c/manager/0.log"
Jul 24 15:15:18.551075 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:18.551043    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-79fb484fd4-72fc9_d48f106a-ac2b-49a6-b4c9-a35de5ffd414/maas-api/0.log"
Jul 24 15:15:19.024866 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:19.024838    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-zbt9r_fcf78e15-bd86-4e8d-82df-941790b7529a/postgres/0.log"
Jul 24 15:15:20.428235 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:20.428212    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5846844f9f-jpfh2_62b14cb5-1877-4259-9464-db6f58c756d8/authorino/0.log"
Jul 24 15:15:20.556141 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:20.556122    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-drndx_4b20f872-19aa-414d-98ff-f907c5cbeba0/manager/0.log"
Jul 24 15:15:20.679156 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:20.679105    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hxgvq_ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c/manager/0.log"
Jul 24 15:15:20.937676 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:20.937632    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rj4tp_761688fb-2c88-46bb-9dfc-5aca73e07a71/registry-server/0.log"
Jul 24 15:15:21.068571 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:21.068551    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-lhp6j_4e243009-02ee-4f91-aa17-36a85a86dd93/manager/0.log"
Jul 24 15:15:21.696422 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:21.696401    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c9pz22_c955ff85-db8b-4764-855b-6b0c4730411c/istio-proxy/0.log"
Jul 24 15:15:22.052478 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:22.052393    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66d67d68d4-r6ngq_5e308560-2a6f-41dc-84d6-f83ffaf7bc66/kube-auth-proxy/0.log"
Jul 24 15:15:22.176170 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:22.176151    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-v8jkd_040f935e-679e-4600-808b-aa9900fd2944/istio-proxy/1.log"
Jul 24 15:15:22.309994 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:22.309947    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-p6w7n_5bc47a37-2e1c-4a32-9536-9944afe9c4b5/payload-pre-processing/0.log"
Jul 24 15:15:22.938341 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:22.938322    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-v8rwq_7e4108b9-bdcc-43b8-856d-89e57fa68901/main/0.log"
Jul 24 15:15:23.179806 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:23.179781    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4jtw5_15db754d-858b-4643-9bb1-0655a0bdb571/main/0.log"
Jul 24 15:15:29.365690 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.365597    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qd87v/must-gather-fmqqg"]
Jul 24 15:15:29.366053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.365971    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="543143f7-190b-46fc-a851-45074397d7af" containerName="payload-processing"
Jul 24 15:15:29.366053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.365988    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="543143f7-190b-46fc-a851-45074397d7af" containerName="payload-processing"
Jul 24 15:15:29.366053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366002    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc4b9bce-5398-41b8-9416-62d9ce2a1e18" containerName="payload-pre-processing"
Jul 24 15:15:29.366053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366008    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc4b9bce-5398-41b8-9416-62d9ce2a1e18" containerName="payload-pre-processing"
Jul 24 15:15:29.366053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366015    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071fff2c-2922-4688-bd3d-c45a286276b5" containerName="maas-api"
Jul 24 15:15:29.366053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366019    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="071fff2c-2922-4688-bd3d-c45a286276b5" containerName="maas-api"
Jul 24 15:15:29.366053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366034    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy"
Jul 24 15:15:29.366053 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366038    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy"
Jul 24 15:15:29.366293 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366093    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc4b9bce-5398-41b8-9416-62d9ce2a1e18" containerName="payload-pre-processing"
Jul 24 15:15:29.366293 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366105    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy"
Jul 24 15:15:29.366293 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366110    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="071fff2c-2922-4688-bd3d-c45a286276b5" containerName="maas-api"
Jul 24 15:15:29.366293 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366116    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy"
Jul 24 15:15:29.366293 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366122    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="543143f7-190b-46fc-a851-45074397d7af" containerName="payload-processing"
Jul 24 15:15:29.366293 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366167    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy"
Jul 24 15:15:29.366293 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.366172    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ceeb8d-8db0-4320-a186-4dd735a0002e" containerName="istio-proxy"
Jul 24 15:15:29.370643 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.370624    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qd87v/must-gather-fmqqg"
Jul 24 15:15:29.374305 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.374287    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qd87v\"/\"kube-root-ca.crt\""
Jul 24 15:15:29.375370 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.375354    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qd87v\"/\"default-dockercfg-m4jjv\""
Jul 24 15:15:29.375424 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.375362    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qd87v\"/\"openshift-service-ca.crt\""
Jul 24 15:15:29.383672 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.383654    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qd87v/must-gather-fmqqg"]
Jul 24 15:15:29.383939 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.383918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:29.459000 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.458981    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5z8g\" (UniqueName: \"kubernetes.io/projected/ac97a48a-2818-46d7-8edf-4817fb3560df-kube-api-access-t5z8g\") pod \"must-gather-fmqqg\" (UID: \"ac97a48a-2818-46d7-8edf-4817fb3560df\") " pod="openshift-must-gather-qd87v/must-gather-fmqqg"
Jul 24 15:15:29.459101 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.459072    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ac97a48a-2818-46d7-8edf-4817fb3560df-must-gather-output\") pod \"must-gather-fmqqg\" (UID: \"ac97a48a-2818-46d7-8edf-4817fb3560df\") " pod="openshift-must-gather-qd87v/must-gather-fmqqg"
Jul 24 15:15:29.478559 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.478536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:29.560039 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.560014    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ac97a48a-2818-46d7-8edf-4817fb3560df-must-gather-output\") pod \"must-gather-fmqqg\" (UID: \"ac97a48a-2818-46d7-8edf-4817fb3560df\") " pod="openshift-must-gather-qd87v/must-gather-fmqqg"
Jul 24 15:15:29.560135 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.560076    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5z8g\" (UniqueName: \"kubernetes.io/projected/ac97a48a-2818-46d7-8edf-4817fb3560df-kube-api-access-t5z8g\") pod \"must-gather-fmqqg\" (UID: \"ac97a48a-2818-46d7-8edf-4817fb3560df\") " pod="openshift-must-gather-qd87v/must-gather-fmqqg"
Jul 24 15:15:29.560307 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.560292    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ac97a48a-2818-46d7-8edf-4817fb3560df-must-gather-output\") pod \"must-gather-fmqqg\" (UID: \"ac97a48a-2818-46d7-8edf-4817fb3560df\") " pod="openshift-must-gather-qd87v/must-gather-fmqqg"
Jul 24 15:15:29.569903 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.569886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5z8g\" (UniqueName: \"kubernetes.io/projected/ac97a48a-2818-46d7-8edf-4817fb3560df-kube-api-access-t5z8g\") pod \"must-gather-fmqqg\" (UID: \"ac97a48a-2818-46d7-8edf-4817fb3560df\") " pod="openshift-must-gather-qd87v/must-gather-fmqqg"
Jul 24 15:15:29.679281 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.679234    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qd87v/must-gather-fmqqg"
Jul 24 15:15:29.810773 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.810744    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qd87v/must-gather-fmqqg"]
Jul 24 15:15:29.813598 ip-10-0-141-133 kubenswrapper[2583]: W0724 15:15:29.813570    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac97a48a_2818_46d7_8edf_4817fb3560df.slice/crio-381424f7b5b5e8105e35f7b4f83aa34ea2d10f3f73252379a2a113a057e63238 WatchSource:0}: Error finding container 381424f7b5b5e8105e35f7b4f83aa34ea2d10f3f73252379a2a113a057e63238: Status 404 returned error can't find the container with id 381424f7b5b5e8105e35f7b4f83aa34ea2d10f3f73252379a2a113a057e63238
Jul 24 15:15:29.815509 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:29.815492    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:15:30.248657 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:30.248630    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qd87v/must-gather-fmqqg" event={"ID":"ac97a48a-2818-46d7-8edf-4817fb3560df","Type":"ContainerStarted","Data":"381424f7b5b5e8105e35f7b4f83aa34ea2d10f3f73252379a2a113a057e63238"}
Jul 24 15:15:31.257060 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:31.256975    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qd87v/must-gather-fmqqg" event={"ID":"ac97a48a-2818-46d7-8edf-4817fb3560df","Type":"ContainerStarted","Data":"6f75b9465e0876b7628ce2044782aef9dea832b518b637424848a77df2f508d5"}
Jul 24 15:15:31.257060 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:31.257022    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qd87v/must-gather-fmqqg" event={"ID":"ac97a48a-2818-46d7-8edf-4817fb3560df","Type":"ContainerStarted","Data":"6b838081f81d5b0c8129aa62fa13a00b24eedcb5fd12b1e2d8aa0c73700eb312"}
Jul 24 15:15:31.275198 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:31.275148    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qd87v/must-gather-fmqqg" podStartSLOduration=1.172985532 podStartE2EDuration="2.275131694s" totalImagesPullingTime="1.102146162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:15:31.273177852 +0000 UTC m=+2516.426178577" watchObservedRunningTime="2026-07-24 15:15:31.275131694 +0000 UTC m=+2516.428132418"
Jul 24 15:15:32.610317 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:32.610290    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lntn6_585bbc0a-c23e-4e78-b089-9e41147e92c9/global-pull-secret-syncer/0.log"
Jul 24 15:15:32.746397 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:32.746371    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dr2g4_4bb9cb9e-5076-44dd-8588-290beecbc993/konnectivity-agent/0.log"
Jul 24 15:15:32.891243 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:32.891156    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-133.ec2.internal_1557375bdf94e26d9bcd144776d91190/haproxy/0.log"
Jul 24 15:15:37.816856 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:37.816815    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5846844f9f-jpfh2_62b14cb5-1877-4259-9464-db6f58c756d8/authorino/0.log"
Jul 24 15:15:37.849248 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:37.849220    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-drndx_4b20f872-19aa-414d-98ff-f907c5cbeba0/manager/0.log"
Jul 24 15:15:37.875944 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:37.875905    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hxgvq_ed6d4a4b-1ff4-4f95-8ca9-658e5c5f963c/manager/0.log"
Jul 24 15:15:37.951300 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:37.951269    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rj4tp_761688fb-2c88-46bb-9dfc-5aca73e07a71/registry-server/0.log"
Jul 24 15:15:38.097567 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:38.096026    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-lhp6j_4e243009-02ee-4f91-aa17-36a85a86dd93/manager/0.log"
Jul 24 15:15:40.238496 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:40.238442    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7r9ws_4f722f0c-651a-4752-91eb-745fb807f5d3/node-exporter/0.log"
Jul 24 15:15:40.267336 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:40.267288    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7r9ws_4f722f0c-651a-4752-91eb-745fb807f5d3/kube-rbac-proxy/0.log"
Jul 24 15:15:40.297678 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:40.297655    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7r9ws_4f722f0c-651a-4752-91eb-745fb807f5d3/init-textfile/0.log"
Jul 24 15:15:41.919921 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:41.919878    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"]
Jul 24 15:15:41.923406 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:41.923375    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:41.937807 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:41.937774    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"]
Jul 24 15:15:41.979285 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:41.979245    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-lib-modules\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:41.979500 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:41.979322    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-proc\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:41.979500 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:41.979371    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-sys\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:41.979500 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:41.979407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-podres\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:41.979500 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:41.979446    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnthn\" (UniqueName: \"kubernetes.io/projected/483b9a20-6d68-4618-899a-704cbca5c2a9-kube-api-access-nnthn\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.080721 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.080685    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-sys\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.080950 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.080742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-podres\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.080950 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.080778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnthn\" (UniqueName: \"kubernetes.io/projected/483b9a20-6d68-4618-899a-704cbca5c2a9-kube-api-access-nnthn\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.080950 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.080824    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-sys\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.080950 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.080834    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-lib-modules\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.080950 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.080907    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-proc\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.081199 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.080956    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-podres\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.081199 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.080979    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-lib-modules\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.081199 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.081006    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/483b9a20-6d68-4618-899a-704cbca5c2a9-proc\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.090654 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.090630    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnthn\" (UniqueName: \"kubernetes.io/projected/483b9a20-6d68-4618-899a-704cbca5c2a9-kube-api-access-nnthn\") pod \"perf-node-gather-daemonset-q6nfj\" (UID: \"483b9a20-6d68-4618-899a-704cbca5c2a9\") " pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.236199 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.236131    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:42.426412 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:42.426339    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"]
Jul 24 15:15:43.343421 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:43.343388    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj" event={"ID":"483b9a20-6d68-4618-899a-704cbca5c2a9","Type":"ContainerStarted","Data":"6a13af1d1ed1d6b7fbddc317c21aaeea092be4d30ba0fb55e4d9fadaa22094b9"}
Jul 24 15:15:43.343778 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:43.343429    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:43.343778 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:43.343473    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj" event={"ID":"483b9a20-6d68-4618-899a-704cbca5c2a9","Type":"ContainerStarted","Data":"644081fa0aa13016817572666c1d3ee473e1d4abafa26d9a73529fc61701338b"}
Jul 24 15:15:43.362902 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:43.362853    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj" podStartSLOduration=2.362837294 podStartE2EDuration="2.362837294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:15:43.361405869 +0000 UTC m=+2528.514406594" watchObservedRunningTime="2026-07-24 15:15:43.362837294 +0000 UTC m=+2528.515838018"
Jul 24 15:15:45.117262 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:45.117233    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z7b8d_04725f4d-6048-4701-a194-7a45ec82c7ea/dns/0.log"
Jul 24 15:15:45.142899 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:45.142877    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z7b8d_04725f4d-6048-4701-a194-7a45ec82c7ea/kube-rbac-proxy/0.log"
Jul 24 15:15:45.165463 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:45.165432    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cxg4g_7429f1ce-5023-470a-9736-077b1ab02ff4/dns-node-resolver/0.log"
Jul 24 15:15:45.804399 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:45.804354    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cr2nk_28c0761e-a2fb-45db-9e96-a0464664e358/node-ca/0.log"
Jul 24 15:15:46.792410 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:46.792359    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c9pz22_c955ff85-db8b-4764-855b-6b0c4730411c/istio-proxy/0.log"
Jul 24 15:15:46.924047 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:46.924008    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66d67d68d4-r6ngq_5e308560-2a6f-41dc-84d6-f83ffaf7bc66/kube-auth-proxy/0.log"
Jul 24 15:15:46.955379 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:46.955352    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-v8jkd_040f935e-679e-4600-808b-aa9900fd2944/istio-proxy/0.log"
Jul 24 15:15:46.987532 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:46.987503    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-v8jkd_040f935e-679e-4600-808b-aa9900fd2944/istio-proxy/1.log"
Jul 24 15:15:47.023716 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:47.023688    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-p6w7n_5bc47a37-2e1c-4a32-9536-9944afe9c4b5/payload-pre-processing/0.log"
Jul 24 15:15:48.108328 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:48.108297    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qddgc_f73843df-70d8-49f8-942e-9b9af5115170/serve-healthcheck-canary/0.log"
Jul 24 15:15:48.757068 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:48.757034    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kwzjl_b9c39353-4f2e-438c-b897-294a4311bf29/kube-rbac-proxy/0.log"
Jul 24 15:15:48.814757 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:48.814731    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kwzjl_b9c39353-4f2e-438c-b897-294a4311bf29/exporter/0.log"
Jul 24 15:15:48.854740 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:48.854714    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kwzjl_b9c39353-4f2e-438c-b897-294a4311bf29/extractor/0.log"
Jul 24 15:15:49.359005 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:49.358977    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qd87v/perf-node-gather-daemonset-q6nfj"
Jul 24 15:15:51.452383 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:51.452316    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c9pz22_c955ff85-db8b-4764-855b-6b0c4730411c/istio-proxy/0.log"
Jul 24 15:15:51.544949 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:51.544920    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c2lfm_43fde56f-fe9b-45a9-a5c5-7f7154cb6183/manager/0.log"
Jul 24 15:15:51.559768 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:51.559743    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66d67d68d4-r6ngq_5e308560-2a6f-41dc-84d6-f83ffaf7bc66/kube-auth-proxy/0.log"
Jul 24 15:15:51.587590 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:51.587543    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-v8jkd_040f935e-679e-4600-808b-aa9900fd2944/istio-proxy/0.log"
Jul 24 15:15:51.645640 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:51.645612    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-v8jkd_040f935e-679e-4600-808b-aa9900fd2944/istio-proxy/1.log"
Jul 24 15:15:51.677213 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:51.677188    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-p6w7n_5bc47a37-2e1c-4a32-9536-9944afe9c4b5/payload-pre-processing/0.log"
Jul 24 15:15:51.738260 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:51.738146    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kr4wf_11622fda-60f5-43aa-b613-ce594d477479/manager/0.log"
Jul 24 15:15:51.855163 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:51.855137    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7955497cfb-b4g79_07e49534-6a68-4595-a3ac-ba93fbfd98e6/manager/0.log"
Jul 24 15:15:51.918316 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:51.918292    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hlsvj_136fc1c8-5b98-4af4-8322-b7fbb686bb39/server/0.log"
Jul 24 15:15:52.023270 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:52.023209    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2dtr8_871167d0-5d94-47d5-8711-52bffbc5c9ae/manager/0.log"
Jul 24 15:15:52.113332 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:52.113303    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-696d5956bc-9m6l4_bc865c7d-dfe1-426b-91ec-f833e581e15c/manager/0.log"
Jul 24 15:15:52.143686 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:52.143658    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-696d5956bc-9m6l4_bc865c7d-dfe1-426b-91ec-f833e581e15c/copy-manifests/0.log"
Jul 24 15:15:52.982350 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:52.982324    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c9pz22_c955ff85-db8b-4764-855b-6b0c4730411c/istio-proxy/0.log"
Jul 24 15:15:53.094883 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:53.094849    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66d67d68d4-r6ngq_5e308560-2a6f-41dc-84d6-f83ffaf7bc66/kube-auth-proxy/0.log"
Jul 24 15:15:53.123480 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:53.123431    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-v8jkd_040f935e-679e-4600-808b-aa9900fd2944/istio-proxy/0.log"
Jul 24 15:15:53.158395 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:53.158362    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-v8jkd_040f935e-679e-4600-808b-aa9900fd2944/istio-proxy/1.log"
Jul 24 15:15:53.184238 ip-10-0-141-133 kubenswrapper[2583]: I0724 15:15:53.184157    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-p6w7n_5bc47a37-2e1c-4a32-9536-9944afe9c4b5/payload-pre-processing/0.log"