Jul 28 12:30:06.943573 ip-10-0-132-90 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 12:30:07.353173 ip-10-0-132-90 kubenswrapper[2549]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 12:30:07.353173 ip-10-0-132-90 kubenswrapper[2549]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 12:30:07.353173 ip-10-0-132-90 kubenswrapper[2549]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 12:30:07.353173 ip-10-0-132-90 kubenswrapper[2549]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 12:30:07.353173 ip-10-0-132-90 kubenswrapper[2549]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 12:30:07.354418 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.353939    2549 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 12:30:07.358312 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358293    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:30:07.358312 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358312    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358316    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358319    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358322    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358325    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358327    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358330    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358334    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358337    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358339    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358349    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358352    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358355    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358358    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358360    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358363    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358367    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358369    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358372    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358374    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:30:07.358382 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358377    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358379    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358382    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358384    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358387    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358390    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358393    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358396    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358400    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358405    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358408    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358411    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358414    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358417    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358419    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358422    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358424    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358427    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358429    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:30:07.358848 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358431    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358436    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358439    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358442    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358445    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358448    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358451    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358453    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358456    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358459    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358462    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358464    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358467    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358469    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358472    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358475    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358478    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358480    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358482    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:30:07.359351 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358485    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358487    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358491    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358493    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358496    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358498    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358501    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358503    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358505    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358508    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358510    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358513    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358515    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358518    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358520    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358522    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358525    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358527    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358531    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:30:07.359817 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358533    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358536    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358538    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358541    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358543    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358546    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358548    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358551    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358555    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358559    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358562    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358565    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358567    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358570    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358572    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358575    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358577    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358580    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358583    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:30:07.360287 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358585    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.358587    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359327    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359335    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359338    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359341    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359343    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359346    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359348    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359351    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359354    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359357    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359360    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359363    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359366    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359369    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359371    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359374    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359376    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359379    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:30:07.360735 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359381    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359384    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359388    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359390    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359393    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359396    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359398    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359400    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359403    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359405    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359408    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359411    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359413    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359416    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359419    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359421    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359424    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359426    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359429    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359431    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:30:07.361242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359434    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359436    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359438    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359441    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359443    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359446    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359451    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359454    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359457    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359460    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359462    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359465    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359467    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359470    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359475    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359479    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359482    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359485    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359488    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:30:07.361742 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359491    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359493    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359496    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359499    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359502    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359506    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359509    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359512    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359515    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359518    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359521    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359523    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359526    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359528    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359531    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359533    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359536    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359539    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359542    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:30:07.362320 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359544    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359547    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359549    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359552    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359555    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359557    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359559    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359563    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359566    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359568    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359571    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359574    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359576    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359578    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359581    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359583    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359585    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359588    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359590    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359593    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:30:07.362773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359595    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359597    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.359600    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360445    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360458    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360462    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360464    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360468    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360470    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360473    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360476    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360480    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360484    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360487    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360490    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360493    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360496    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360498    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360501    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:30:07.363313 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360504    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360507    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360509    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360512    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360514    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360517    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360520    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360523    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360525    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360527    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360530    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360533    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360535    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360538    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360540    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360542    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360544    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360547    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360549    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360566    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:30:07.363780 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360570    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360573    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360576    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360579    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360582    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360585    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360588    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360591    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360594    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360597    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360600    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360602    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360605    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360608    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360611    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360614    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360616    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360618    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360621    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360624    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:30:07.364271 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360626    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360628    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360631    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360633    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360635    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360638    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360641    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360644    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360647    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360651    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360654    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360656    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360659    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360661    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360663    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360666    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360669    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360671    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360674    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:30:07.364773 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360677    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360679    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360682    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360684    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360687    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360689    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360692    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360694    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360697    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360699    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360702    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360704    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360707    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360710    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360712    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360715    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360717    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360720    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360722    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360724    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:30:07.365242 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360727    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360729    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360733    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.360736    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360808    2549 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360815    2549 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360822    2549 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360831    2549 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360836    2549 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360842    2549 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360846    2549 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360850    2549 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360854    2549 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360857    2549 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360860    2549 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360864    2549 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360867    2549 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360870    2549 flags.go:64] FLAG: --cgroup-root=""
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360873    2549 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360876    2549 flags.go:64] FLAG: --client-ca-file=""
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360878    2549 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360881    2549 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360885    2549 flags.go:64] FLAG: --cluster-domain=""
Jul 28 12:30:07.365738 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360888    2549 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360891    2549 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360894    2549 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360897    2549 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360901    2549 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360903    2549 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360906    2549 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360910    2549 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360913    2549 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360915    2549 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360918    2549 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360921    2549 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360924    2549 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360928    2549 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360931    2549 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360934    2549 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360937    2549 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360940    2549 flags.go:64] FLAG: --enable-server="true"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360944    2549 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360947    2549 flags.go:64] FLAG: --event-burst="100"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360951    2549 flags.go:64] FLAG: --event-qps="50"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360953    2549 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360956    2549 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360959    2549 flags.go:64] FLAG: --eviction-hard=""
Jul 28 12:30:07.366346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360964    2549 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360966    2549 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360970    2549 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360973    2549 flags.go:64] FLAG: --eviction-soft=""
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360975    2549 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360978    2549 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360981    2549 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360984    2549 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360987    2549 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360989    2549 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360992    2549 flags.go:64] FLAG: --feature-gates=""
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360995    2549 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.360998    2549 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361001    2549 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361004    2549 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361007    2549 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361010    2549 flags.go:64] FLAG: --help="false"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361013    2549 flags.go:64] FLAG: --hostname-override="ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361016    2549 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361018    2549 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361021    2549 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361024    2549 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361028    2549 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361031    2549 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 12:30:07.366941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361034    2549 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361037    2549 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361041    2549 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361044    2549 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361047    2549 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361050    2549 flags.go:64] FLAG: --kube-reserved=""
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361053    2549 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361055    2549 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361058    2549 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361062    2549 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361065    2549 flags.go:64] FLAG: --lock-file=""
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361067    2549 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361071    2549 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361073    2549 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361079    2549 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361082    2549 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361085    2549 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361087    2549 flags.go:64] FLAG: --logging-format="text"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361090    2549 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361093    2549 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361096    2549 flags.go:64] FLAG: --manifest-url=""
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361099    2549 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361104    2549 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361106    2549 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361111    2549 flags.go:64] FLAG: --max-pods="110"
Jul 28 12:30:07.367658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361114    2549 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361116    2549 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361119    2549 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361122    2549 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361125    2549 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361128    2549 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361131    2549 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361138    2549 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361140    2549 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361145    2549 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361147    2549 flags.go:64] FLAG: --pod-cidr=""
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361150    2549 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361156    2549 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361159    2549 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361162    2549 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361165    2549 flags.go:64] FLAG: --port="10250"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361168    2549 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361171    2549 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03c4e0bbc9143e33e"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361174    2549 flags.go:64] FLAG: --qos-reserved=""
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361177    2549 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361181    2549 flags.go:64] FLAG: --register-node="true"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361183    2549 flags.go:64] FLAG: --register-with-taints=""
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361187    2549 flags.go:64] FLAG: --registry-burst="10"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361190    2549 flags.go:64] FLAG: --registry-qps="5"
Jul 28 12:30:07.368602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361192    2549 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361195    2549 flags.go:64] FLAG: --reserved-memory=""
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361198    2549 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361201    2549 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361204    2549 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361207    2549 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361210    2549 flags.go:64] FLAG: --runonce="false"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361213    2549 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361216    2549 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361219    2549 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361235    2549 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361238    2549 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361241    2549 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361244    2549 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361249    2549 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361252    2549 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361255    2549 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361258    2549 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361261    2549 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361264    2549 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361267    2549 flags.go:64] FLAG: --system-cgroups=""
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361270    2549 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361275    2549 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361278    2549 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361281    2549 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 12:30:07.369271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361285    2549 flags.go:64] FLAG: --tls-min-version=""
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361288    2549 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361291    2549 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361294    2549 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361296    2549 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361299    2549 flags.go:64] FLAG: --v="2"
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361303    2549 flags.go:64] FLAG: --version="false"
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361307    2549 flags.go:64] FLAG: --vmodule=""
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361310    2549 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361313    2549 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361432    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361435    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361439    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361442    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361444    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361447    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361450    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361452    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361455    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361457    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361460    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361464    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:30:07.369899 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361467    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361469    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361472    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361475    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361478    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361480    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361483    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361486    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361488    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361491    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361494    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361497    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361500    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361502    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361505    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361507    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361510    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361512    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361514    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361517    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:30:07.370481 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361520    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361524    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361527    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361530    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361532    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361535    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361538    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361541    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361543    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361546    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361548    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361552    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361554    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361557    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361559    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361561    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361564    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361566    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361570    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:30:07.370954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361574    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361576    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361579    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361582    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361586    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361589    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361592    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361594    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361596    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361599    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361601    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361603    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361606    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361608    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361610    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361613    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361615    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361617    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361620    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:30:07.371483 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361622    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361624    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361626    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361629    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361631    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361635    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361638    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361640    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361642    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361645    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361647    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361649    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361652    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361654    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361657    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361659    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361662    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361664    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361666    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361669    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:30:07.371932 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361671    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361674    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361676    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361679    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361681    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361683    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361686    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361688    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.361691    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.361696    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.371945    2549 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.371967    2549 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.371987    2549 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.371993    2549 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372100    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:30:07.372436 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372105    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372112    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372115    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372119    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372121    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372124    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372127    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372130    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372133    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372135    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372138    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372140    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372144    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372150    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372153    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372156    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372158    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372162    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372164    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:30:07.372812 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372167    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372177    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372180    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372183    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372186    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372188    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372194    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372196    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372199    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372202    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372205    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372208    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372211    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372214    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372216    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372219    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372239    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372252    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372255    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372260    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:30:07.373292 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372263    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372266    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372268    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372271    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372273    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372276    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372278    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372283    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372287    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372290    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372293    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372298    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372301    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372304    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372313    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372316    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372319    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372322    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372325    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:30:07.373799 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372328    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372330    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372333    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372336    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372339    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372343    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372346    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372349    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372352    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372354    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372357    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372359    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372362    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372365    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372367    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372369    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372372    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372375    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372380    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372383    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:30:07.374280 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372386    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372388    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372391    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372393    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372396    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372398    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372401    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372404    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372413    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372417    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372423    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372425    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372428    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372431    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372433    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372436    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372438    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372441    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372444    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:30:07.374767 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372447    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.372453    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372709    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372714    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372717    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372723    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372726    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372729    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372732    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372735    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372737    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372740    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372743    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372746    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372749    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 12:30:07.375233 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372751    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372754    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372756    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372762    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372764    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372767    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372769    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372777    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372780    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372783    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372786    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372788    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372791    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372793    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372796    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372801    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372804    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372807    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372810    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372812    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 12:30:07.375605 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372815    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372818    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372821    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372824    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372826    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372829    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372831    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372835    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372842    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372845    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372848    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372851    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372854    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372856    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372859    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372861    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372864    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372866    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372869    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372871    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 12:30:07.376098 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372883    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372885    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372888    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372891    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372893    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372896    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372898    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372901    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372903    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372906    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372908    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372911    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372913    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372918    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372921    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372923    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372926    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372930    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372933    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 12:30:07.376588 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372936    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372939    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372941    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372944    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372947    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372949    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372952    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372957    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372960    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372963    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372966    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372970    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372973    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372976    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372984    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372987    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372990    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372992    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372994    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.372999    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 12:30:07.377050 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.373003    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 12:30:07.377626 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.373005    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 12:30:07.377626 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.373008    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 12:30:07.377626 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.373010    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 12:30:07.377626 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.373013    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 12:30:07.377626 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.373016    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 12:30:07.377626 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:07.373018    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 12:30:07.377626 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.373023    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 12:30:07.377626 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.373248    2549 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 12:30:07.377626 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.377012    2549 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 12:30:07.378515 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.378498    2549 server.go:1019] "Starting client certificate rotation"
Jul 28 12:30:07.378623 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.378604    2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 12:30:07.378659 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.378642    2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 12:30:07.382077 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.382061    2549 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 12:30:07.386271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.386251    2549 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 12:30:07.396194 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.396170    2549 log.go:25] "Validated CRI v1 runtime API"
Jul 28 12:30:07.399400 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.399383    2549 log.go:25] "Validated CRI v1 image API"
Jul 28 12:30:07.400467 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.400449    2549 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 12:30:07.402787 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.402767    2549 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a609f404-a4c1-4c6b-a939-5c46f2d039d0:/dev/nvme0n1p4 ca0940e9-186a-4f99-b92e-d59e74e130ca:/dev/nvme0n1p3]
Jul 28 12:30:07.402867 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.402787    2549 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 12:30:07.408289 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.408265    2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 12:30:07.408386 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.408249    2549 manager.go:216] Machine: {Timestamp:2026-07-28 12:30:07.406926001 +0000 UTC m=+0.343574963 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099743 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208b232ce56c0cb5ffdd31afb81722 SystemUUID:ec208b23-2ce5-6c0c-b5ff-dd31afb81722 BootID:19578e37-3876-4ae1-801e-e175bb3e2569 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:33:73:57:4e:85 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:33:73:57:4e:85 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:35:36:1d:8b:3e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 12:30:07.408386 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.408369    2549 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 12:30:07.408489 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.408476    2549 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 12:30:07.411452 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.411413    2549 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 12:30:07.411600 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.411457    2549 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-90.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 12:30:07.412129 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.412119    2549 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 12:30:07.412162 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.412132    2549 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 12:30:07.412162 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.412141    2549 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 12:30:07.412162 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.412155    2549 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 12:30:07.413041 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.413030    2549 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 12:30:07.414850 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.414837    2549 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 12:30:07.414978 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.414968    2549 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 12:30:07.415649 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.415638    2549 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 12:30:07.415693 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.415653    2549 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 12:30:07.415693 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.415669    2549 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 12:30:07.415693 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.415678    2549 kubelet.go:398] "Adding apiserver pod source"
Jul 28 12:30:07.415693 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.415687    2549 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 12:30:07.416104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.416085    2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 12:30:07.416104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.416108    2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 12:30:07.420213 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.420190    2549 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 12:30:07.422133 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.422118    2549 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 12:30:07.422186 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.422150    2549 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 12:30:07.423611 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423600    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 12:30:07.423658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423617    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 12:30:07.423658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423623    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 12:30:07.423658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423629    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 12:30:07.423658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423639    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 12:30:07.423658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423648    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 12:30:07.423658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423657    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 12:30:07.423840 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423662    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 12:30:07.423840 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423669    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 12:30:07.423840 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423674    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 12:30:07.423840 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423683    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 12:30:07.423840 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.423688    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 12:30:07.425198 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.425188    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 12:30:07.425198 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.425197    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 12:30:07.427534 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.427514    2549 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-90.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 12:30:07.427630 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.427529    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-90.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 12:30:07.427768 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.427721    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 12:30:07.428526 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.428514    2549 server.go:1268] "Started kubelet"
Jul 28 12:30:07.428624 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.428586    2549 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 12:30:07.428691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.428596    2549 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 12:30:07.428691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.428647    2549 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 12:30:07.429455 ip-10-0-132-90 systemd[1]: Started Kubernetes Kubelet.
Jul 28 12:30:07.430417 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.430401    2549 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 12:30:07.430654 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.430643    2549 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 12:30:07.430886 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.430866    2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-88qz4"
Jul 28 12:30:07.435446 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.435419    2549 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 12:30:07.436104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.436088    2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-88qz4"
Jul 28 12:30:07.436757 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.435898    2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-90.ec2.internal.18c673f719f37d7b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-90.ec2.internal,UID:ip-10-0-132-90.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-90.ec2.internal,},FirstTimestamp:2026-07-28 12:30:07.428484475 +0000 UTC m=+0.365133438,LastTimestamp:2026-07-28 12:30:07.428484475 +0000 UTC m=+0.365133438,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-90.ec2.internal,}"
Jul 28 12:30:07.437657 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.437642    2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 12:30:07.437714 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.437646    2549 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 12:30:07.437821 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.437806    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:07.438269 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438251    2549 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 12:30:07.438269 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438263    2549 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 12:30:07.438404 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438282    2549 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 12:30:07.438532 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438502    2549 factory.go:55] Registering systemd factory
Jul 28 12:30:07.438532 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438525    2549 factory.go:223] Registration of the systemd container factory successfully
Jul 28 12:30:07.438758 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438744    2549 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 12:30:07.438758 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438760    2549 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 12:30:07.438849 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438784    2549 factory.go:153] Registering CRI-O factory
Jul 28 12:30:07.438849 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438833    2549 factory.go:223] Registration of the crio container factory successfully
Jul 28 12:30:07.438940 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438882    2549 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 12:30:07.438940 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438902    2549 factory.go:103] Registering Raw factory
Jul 28 12:30:07.438940 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.438913    2549 manager.go:1196] Started watching for new ooms in manager
Jul 28 12:30:07.439602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.439588    2549 manager.go:349] Starting recovery of all containers
Jul 28 12:30:07.440250 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.440200    2549 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-90.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 12:30:07.440362 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.440322    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 12:30:07.451262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.451218    2549 manager.go:354] Recovery completed
Jul 28 12:30:07.452759 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.452732    2549 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 12:30:07.455960 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.455945    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:30:07.457975 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.457951    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:30:07.458063 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.457992    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:30:07.458063 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.458004    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:30:07.458711 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.458693    2549 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 12:30:07.458711 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.458702    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:30:07.458711 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.458709    2549 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 12:30:07.458876 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.458726    2549 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 12:30:07.459584 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.459438    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:30:07.459644 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.459598    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:30:07.459644 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.459610    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:30:07.460759 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.460748    2549 policy_none.go:49] "None policy: Start"
Jul 28 12:30:07.460797 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.460762    2549 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 12:30:07.460797 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.460772    2549 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 12:30:07.461812 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.461800    2549 policy_none.go:47] "Start"
Jul 28 12:30:07.505970 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.505948    2549 manager.go:337] "Starting Device Plugin manager"
Jul 28 12:30:07.506107 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.506010    2549 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 12:30:07.506107 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.506024    2549 server.go:87] "Starting device plugin registration server"
Jul 28 12:30:07.506374 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.506359    2549 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 12:30:07.506447 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.506376    2549 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 12:30:07.506492 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.506477    2549 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 12:30:07.506600 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.506564    2549 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 12:30:07.506600 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.506576    2549 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 12:30:07.507179 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.507148    2549 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 12:30:07.507265 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.507245    2549 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:07.607538 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.607461    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:30:07.608606 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.608541    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:30:07.608606 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.608574    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:30:07.608606 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.608584    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:30:07.608723 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.608614    2549 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.611500 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.611472    2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 12:30:07.612806 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.612773    2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 12:30:07.612892 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.612812    2549 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 12:30:07.612892 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.612837    2549 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 12:30:07.612892 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.612870    2549 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 12:30:07.615701 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.615682    2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:30:07.617181 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.617164    2549 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.617293 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.617189    2549 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-90.ec2.internal\": node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:07.646111 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.646079    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:07.713772 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.713717    2549 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal"]
Jul 28 12:30:07.713871 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.713826    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:30:07.714866 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.714846    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:30:07.714972 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.714883    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:30:07.714972 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.714895    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:30:07.716335 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.716323    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:30:07.716458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.716445    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.716494 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.716475    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:30:07.717704 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.717680    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:30:07.717788 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.717709    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:30:07.717788 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.717721    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:30:07.717788 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.717685    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:30:07.717788 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.717772    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:30:07.717788 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.717781    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:30:07.718940 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.718928    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.718984 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.718950    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 12:30:07.719836 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.719816    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 12:30:07.719914 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.719844    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 12:30:07.719914 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.719858    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeHasSufficientPID"
Jul 28 12:30:07.739759 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.739724    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9415a092b6cee87f2b0f814e0eb8bce0-config\") pod \"kube-apiserver-proxy-ip-10-0-132-90.ec2.internal\" (UID: \"9415a092b6cee87f2b0f814e0eb8bce0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.739759 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.739759    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ec3e761d1583b4f7566737026b3db1f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal\" (UID: \"3ec3e761d1583b4f7566737026b3db1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.739943 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.739782    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ec3e761d1583b4f7566737026b3db1f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal\" (UID: \"3ec3e761d1583b4f7566737026b3db1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.744110 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.744094    2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-90.ec2.internal\" not found" node="ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.746189 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.746169    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:07.748529 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.748512    2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-90.ec2.internal\" not found" node="ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.840470 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.840437    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ec3e761d1583b4f7566737026b3db1f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal\" (UID: \"3ec3e761d1583b4f7566737026b3db1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.840666 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.840492    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ec3e761d1583b4f7566737026b3db1f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal\" (UID: \"3ec3e761d1583b4f7566737026b3db1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.840666 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.840517    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9415a092b6cee87f2b0f814e0eb8bce0-config\") pod \"kube-apiserver-proxy-ip-10-0-132-90.ec2.internal\" (UID: \"9415a092b6cee87f2b0f814e0eb8bce0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.840666 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.840526    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ec3e761d1583b4f7566737026b3db1f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal\" (UID: \"3ec3e761d1583b4f7566737026b3db1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.840666 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.840550    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9415a092b6cee87f2b0f814e0eb8bce0-config\") pod \"kube-apiserver-proxy-ip-10-0-132-90.ec2.internal\" (UID: \"9415a092b6cee87f2b0f814e0eb8bce0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.840666 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:07.840613    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ec3e761d1583b4f7566737026b3db1f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal\" (UID: \"3ec3e761d1583b4f7566737026b3db1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:07.846401 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:07.846368    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.019913 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.019878    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.119444 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.119409    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:09.120509 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.120482    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.124366 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.124350    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:09.221449 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.221351    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.322021 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.321988    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.422672 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.422607    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.450789 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.450755    2549 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 12:30:09.451235 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.450905    2549 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 12:30:09.513058 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.512982    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 12:25:08 +0000 UTC" deadline="2028-02-04 21:19:04.252010956 +0000 UTC"
Jul 28 12:30:09.513058 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.513012    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13352h48m54.739002086s"
Jul 28 12:30:09.522903 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.522879    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.623369 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.623336    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.630920 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:09.630875    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9415a092b6cee87f2b0f814e0eb8bce0.slice/crio-026bb182580fe9e72b733b7e16c9f057413f21691cdaa5e9fb232609f63c3907 WatchSource:0}: Error finding container 026bb182580fe9e72b733b7e16c9f057413f21691cdaa5e9fb232609f63c3907: Status 404 returned error can't find the container with id 026bb182580fe9e72b733b7e16c9f057413f21691cdaa5e9fb232609f63c3907
Jul 28 12:30:09.631704 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:09.631665    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec3e761d1583b4f7566737026b3db1f.slice/crio-8f44c070c59f07285fc8ecc46777d15d6e2696668e64f7ba86fd4ac5ee8e7ae1 WatchSource:0}: Error finding container 8f44c070c59f07285fc8ecc46777d15d6e2696668e64f7ba86fd4ac5ee8e7ae1: Status 404 returned error can't find the container with id 8f44c070c59f07285fc8ecc46777d15d6e2696668e64f7ba86fd4ac5ee8e7ae1
Jul 28 12:30:09.631830 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.631700    2549 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:30:09.636823 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.636803    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 12:30:09.689588 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.689525    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal" event={"ID":"3ec3e761d1583b4f7566737026b3db1f","Type":"ContainerStarted","Data":"8f44c070c59f07285fc8ecc46777d15d6e2696668e64f7ba86fd4ac5ee8e7ae1"}
Jul 28 12:30:09.690619 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.690597    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal" event={"ID":"9415a092b6cee87f2b0f814e0eb8bce0","Type":"ContainerStarted","Data":"026bb182580fe9e72b733b7e16c9f057413f21691cdaa5e9fb232609f63c3907"}
Jul 28 12:30:09.724101 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.724068    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.824654 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.824554    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:09.865672 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:09.865646    2549 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:30:09.925653 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:09.925605    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:10.026544 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.026508    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-90.ec2.internal\" not found"
Jul 28 12:30:10.089020 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.088940    2549 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:30:10.110316 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.110283    2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:10.121615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.121583    2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 12:30:10.122493 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.122467    2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal"
Jul 28 12:30:10.133735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.133698    2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 12:30:10.269970 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.269934    2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 12:30:10.490040 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.489982    2549 apiserver.go:52] "Watching apiserver"
Jul 28 12:30:10.503637 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.503593    2549 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 12:30:10.504069 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.504049    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-7tctq","kube-system/global-pull-secret-syncer-sctjj","kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal","openshift-cluster-node-tuning-operator/tuned-lp6n8","openshift-dns/node-resolver-bhst2","openshift-image-registry/node-ca-qbgl6","openshift-multus/multus-28hw7","openshift-network-diagnostics/network-check-target-v7b58","openshift-network-operator/iptables-alerter-hqmpp","kube-system/konnectivity-agent-z7jj4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal","openshift-multus/multus-additional-cni-plugins-j7655","openshift-multus/network-metrics-daemon-mg9wl"]
Jul 28 12:30:10.505617 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.505600    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.506904 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.506883    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.508026 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.508006    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:10.508116 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.508083    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:10.508609 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.508590    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5chlh\""
Jul 28 12:30:10.508924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.508907    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 12:30:10.509044 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.508936    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 12:30:10.509044 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.508944    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 12:30:10.509044 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.508981    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 12:30:10.509247 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.509236    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.510367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.510345    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 12:30:10.510475 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.510372    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bzb22\""
Jul 28 12:30:10.510475 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.510466    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 12:30:10.510843 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.510826    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.511386 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.510923    2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 12:30:10.512188 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.512157    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.512280 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.512222    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 12:30:10.515869 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.513008    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 12:30:10.515869 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.513124    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mlvwk\""
Jul 28 12:30:10.515869 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.513808    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xmdhq\""
Jul 28 12:30:10.515869 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.514103    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 12:30:10.515869 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.514372    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.515869 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.514856    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 12:30:10.515869 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.515044    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 12:30:10.516245 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.515985    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4j87x\""
Jul 28 12:30:10.516245 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.516127    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 12:30:10.516381 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.516361    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 12:30:10.516805 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.516787    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.517819 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.517714    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 12:30:10.517819 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.517758    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vqb5p\""
Jul 28 12:30:10.517819 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.517813    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 12:30:10.518004 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.517859    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 12:30:10.518004 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.517997    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:10.518097 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.518067    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:10.519357 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.519334    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:10.519925 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.519907    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 12:30:10.520273 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.520256    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 12:30:10.521120 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.521106    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.521410 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.521388    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 12:30:10.522040 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.521784    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 12:30:10.522040 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.521783    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 12:30:10.522040 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.521818    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 12:30:10.522040 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.521884    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8678p\""
Jul 28 12:30:10.522405 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522386    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 12:30:10.522505 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522450    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gj5rb\""
Jul 28 12:30:10.522505 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522463    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:10.522683 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.522522    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:10.522749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522689    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-host\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.522749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522714    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-os-release\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.522749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522737    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-sysctl-d\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.522945 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522766    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-tuned\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.522945 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522801    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wmjp\" (UniqueName: \"kubernetes.io/projected/d917c83e-d627-4099-ba45-58674437b5a8-kube-api-access-5wmjp\") pod \"node-resolver-bhst2\" (UID: \"d917c83e-d627-4099-ba45-58674437b5a8\") " pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.522945 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522828    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-multus-cni-dir\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.522945 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522852    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-run-netns\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.522945 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522890    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-multus-conf-dir\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.522945 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522917    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc7866d7-9027-408c-a015-d30a286aede0-multus-daemon-config\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.522963    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-run\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523002    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-var-lib-kubelet\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523031    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d917c83e-d627-4099-ba45-58674437b5a8-hosts-file\") pod \"node-resolver-bhst2\" (UID: \"d917c83e-d627-4099-ba45-58674437b5a8\") " pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523056    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1b592e4-fbbf-4fe8-aa78-c10e97343d81-host\") pod \"node-ca-qbgl6\" (UID: \"f1b592e4-fbbf-4fe8-aa78-c10e97343d81\") " pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523083    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc7866d7-9027-408c-a015-d30a286aede0-cni-binary-copy\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523094    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523108    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-multus-socket-dir-parent\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523133    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-var-lib-cni-multus\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523156    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-hostroot\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523178    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-etc-kubernetes\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523219    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-cnibin\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523241    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-cni-binary-copy\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.523262 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523263    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-run-multus-certs\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523285    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-system-cni-dir\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523307    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-kubernetes\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523325    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-systemd\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523359    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-lib-modules\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523395    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d917c83e-d627-4099-ba45-58674437b5a8-tmp-dir\") pod \"node-resolver-bhst2\" (UID: \"d917c83e-d627-4099-ba45-58674437b5a8\") " pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523422    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kswwc\" (UniqueName: \"kubernetes.io/projected/f1b592e4-fbbf-4fe8-aa78-c10e97343d81-kube-api-access-kswwc\") pod \"node-ca-qbgl6\" (UID: \"f1b592e4-fbbf-4fe8-aa78-c10e97343d81\") " pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523446    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-sysctl-conf\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523468    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1b592e4-fbbf-4fe8-aa78-c10e97343d81-serviceca\") pod \"node-ca-qbgl6\" (UID: \"f1b592e4-fbbf-4fe8-aa78-c10e97343d81\") " pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523505    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-cnibin\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523528    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgl5m\" (UniqueName: \"kubernetes.io/projected/cc7866d7-9027-408c-a015-d30a286aede0-kube-api-access-bgl5m\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523553    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523576    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523598    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-sys\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523635    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66007fee-c832-4f45-8cd7-e9759fe3f3f8-tmp\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523660    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4tm6\" (UniqueName: \"kubernetes.io/projected/66007fee-c832-4f45-8cd7-e9759fe3f3f8-kube-api-access-l4tm6\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523695    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-system-cni-dir\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.523987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523735    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-var-lib-cni-bin\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523769    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-os-release\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523896    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jksd\" (UniqueName: \"kubernetes.io/projected/5a25858a-ae63-4104-b1a4-33ac27e882df-kube-api-access-9jksd\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523931    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-whbjj\""
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523925    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-sysconfig\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523964    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-run-k8s-cni-cncf-io\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.523988    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-var-lib-kubelet\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.524011    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.524049    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.524095    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmd4j\" (UniqueName: \"kubernetes.io/projected/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-kube-api-access-gmd4j\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.524120    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-modprobe-d\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.524604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.524150    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 12:30:10.525139 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.524857    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 12:30:10.525139 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.524873    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 12:30:10.529248 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.529226    2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 12:30:10.558171 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.558142    2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-64t5w"
Jul 28 12:30:10.566691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.566664    2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-64t5w"
Jul 28 12:30:10.611749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.611715    2549 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 12:30:10.625268 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625231    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-etc-openvswitch\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625275    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-run-ovn\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625304    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-multus-socket-dir-parent\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625321    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-var-lib-cni-multus\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625342    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-cni-binary-copy\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625358    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-sys-fs\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625372    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4vq8\" (UniqueName: \"kubernetes.io/projected/c9b36391-2125-47e3-80f5-4c880c52d2a3-kube-api-access-s4vq8\") pod \"iptables-alerter-hqmpp\" (UID: \"c9b36391-2125-47e3-80f5-4c880c52d2a3\") " pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625387    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-run-openvswitch\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625409    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xksb4\" (UniqueName: \"kubernetes.io/projected/3a86f337-8566-420f-aab9-5d9329d8982c-kube-api-access-xksb4\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625401    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-multus-socket-dir-parent\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.625458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625447    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-run-multus-certs\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625470    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-var-lib-cni-multus\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625509    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-system-cni-dir\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625527    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-run-multus-certs\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625553    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-system-cni-dir\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625582    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-kubernetes\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625603    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kswwc\" (UniqueName: \"kubernetes.io/projected/f1b592e4-fbbf-4fe8-aa78-c10e97343d81-kube-api-access-kswwc\") pod \"node-ca-qbgl6\" (UID: \"f1b592e4-fbbf-4fe8-aa78-c10e97343d81\") " pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625653    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cee404ae-196d-4765-8dad-73a8c1da3793-agent-certs\") pod \"konnectivity-agent-z7jj4\" (UID: \"cee404ae-196d-4765-8dad-73a8c1da3793\") " pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625680    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-kubernetes\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625751    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b4434036-47bd-45c9-b08b-12ab9f5f84c6-kubelet-config\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625780    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625801    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-systemd-units\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625827    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-var-lib-openvswitch\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625843    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a86f337-8566-420f-aab9-5d9329d8982c-ovn-node-metrics-cert\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625886    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.625962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.625916    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.626034    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626083    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4tm6\" (UniqueName: \"kubernetes.io/projected/66007fee-c832-4f45-8cd7-e9759fe3f3f8-kube-api-access-l4tm6\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.626127    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:30:11.126086788 +0000 UTC m=+2.990336626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626127    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626148    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626175    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3a86f337-8566-420f-aab9-5d9329d8982c-ovnkube-config\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626176    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-cni-binary-copy\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626217    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-var-lib-cni-bin\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626268    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jksd\" (UniqueName: \"kubernetes.io/projected/5a25858a-ae63-4104-b1a4-33ac27e882df-kube-api-access-9jksd\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626291    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-node-log\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626316    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626338    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmd4j\" (UniqueName: \"kubernetes.io/projected/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-kube-api-access-gmd4j\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626357    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-modprobe-d\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626376    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-host\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626397    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-tuned\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.626615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626414    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-socket-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626429    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbf4w\" (UniqueName: \"kubernetes.io/projected/c224b479-8deb-459f-a0f9-565bf7aba451-kube-api-access-sbf4w\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626453    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wmjp\" (UniqueName: \"kubernetes.io/projected/d917c83e-d627-4099-ba45-58674437b5a8-kube-api-access-5wmjp\") pod \"node-resolver-bhst2\" (UID: \"d917c83e-d627-4099-ba45-58674437b5a8\") " pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626484    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626504    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-run-systemd\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626520    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-log-socket\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626538    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b4434036-47bd-45c9-b08b-12ab9f5f84c6-dbus\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626555    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-multus-cni-dir\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626571    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-multus-conf-dir\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626586    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-run\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626601    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-var-lib-kubelet\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626637    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-slash\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626659    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc7866d7-9027-408c-a015-d30a286aede0-cni-binary-copy\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626676    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-hostroot\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626692    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-etc-kubernetes\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626714    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-cnibin\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626724    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-modprobe-d\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.627367 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626825    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-lib-modules\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626729    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-lib-modules\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626865    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-etc-selinux\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626881    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626907    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-systemd\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626909    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626925    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d917c83e-d627-4099-ba45-58674437b5a8-tmp-dir\") pod \"node-resolver-bhst2\" (UID: \"d917c83e-d627-4099-ba45-58674437b5a8\") " pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626953    2549 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626961    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-registration-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626995    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-device-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627022    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c9b36391-2125-47e3-80f5-4c880c52d2a3-iptables-alerter-script\") pod \"iptables-alerter-hqmpp\" (UID: \"c9b36391-2125-47e3-80f5-4c880c52d2a3\") " pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627045    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-cni-bin\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627070    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627103    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-sysctl-conf\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627125    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d917c83e-d627-4099-ba45-58674437b5a8-tmp-dir\") pod \"node-resolver-bhst2\" (UID: \"d917c83e-d627-4099-ba45-58674437b5a8\") " pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627128    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1b592e4-fbbf-4fe8-aa78-c10e97343d81-serviceca\") pod \"node-ca-qbgl6\" (UID: \"f1b592e4-fbbf-4fe8-aa78-c10e97343d81\") " pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627153    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3a86f337-8566-420f-aab9-5d9329d8982c-env-overrides\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.628087 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627180    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-cnibin\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627184    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-systemd\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627212    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgl5m\" (UniqueName: \"kubernetes.io/projected/cc7866d7-9027-408c-a015-d30a286aede0-kube-api-access-bgl5m\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627235    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-sys\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627244    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-cnibin\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.626266    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-var-lib-cni-bin\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627256    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66007fee-c832-4f45-8cd7-e9759fe3f3f8-tmp\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627289    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cee404ae-196d-4765-8dad-73a8c1da3793-konnectivity-ca\") pod \"konnectivity-agent-z7jj4\" (UID: \"cee404ae-196d-4765-8dad-73a8c1da3793\") " pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627288    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-multus-cni-dir\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627329    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-system-cni-dir\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627336    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-host\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627405    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-multus-conf-dir\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627418    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-hostroot\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627420    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-sys\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627425    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-os-release\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627462    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-cnibin\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627464    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-sysconfig\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627480    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-var-lib-kubelet\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.628735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627503    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-etc-kubernetes\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627529    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-system-cni-dir\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627548    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-os-release\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627564    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-sysconfig\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627586    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-run\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627645    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9b36391-2125-47e3-80f5-4c880c52d2a3-host-slash\") pod \"iptables-alerter-hqmpp\" (UID: \"c9b36391-2125-47e3-80f5-4c880c52d2a3\") " pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627680    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-cni-netd\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627677    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-sysctl-conf\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627712    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3a86f337-8566-420f-aab9-5d9329d8982c-ovnkube-script-lib\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627777    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-run-k8s-cni-cncf-io\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627805    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-run-k8s-cni-cncf-io\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627842    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-var-lib-kubelet\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627898    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627904    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-var-lib-kubelet\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627927    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1b592e4-fbbf-4fe8-aa78-c10e97343d81-serviceca\") pod \"node-ca-qbgl6\" (UID: \"f1b592e4-fbbf-4fe8-aa78-c10e97343d81\") " pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627949    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-os-release\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.627976    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-sysctl-d\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.629548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628004    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-kubelet\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628025    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-os-release\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628040    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-run-netns\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628073    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc7866d7-9027-408c-a015-d30a286aede0-multus-daemon-config\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628101    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc7866d7-9027-408c-a015-d30a286aede0-cni-binary-copy\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628173    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc7866d7-9027-408c-a015-d30a286aede0-host-run-netns\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628194    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d917c83e-d627-4099-ba45-58674437b5a8-hosts-file\") pod \"node-resolver-bhst2\" (UID: \"d917c83e-d627-4099-ba45-58674437b5a8\") " pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628277    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1b592e4-fbbf-4fe8-aa78-c10e97343d81-host\") pod \"node-ca-qbgl6\" (UID: \"f1b592e4-fbbf-4fe8-aa78-c10e97343d81\") " pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628325    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-run-netns\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628343    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d917c83e-d627-4099-ba45-58674437b5a8-hosts-file\") pod \"node-resolver-bhst2\" (UID: \"d917c83e-d627-4099-ba45-58674437b5a8\") " pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628428    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1b592e4-fbbf-4fe8-aa78-c10e97343d81-host\") pod \"node-ca-qbgl6\" (UID: \"f1b592e4-fbbf-4fe8-aa78-c10e97343d81\") " pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628515    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-sysctl-d\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628610    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc7866d7-9027-408c-a015-d30a286aede0-multus-daemon-config\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.628662    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.630323    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66007fee-c832-4f45-8cd7-e9759fe3f3f8-etc-tuned\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.630398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.630349    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66007fee-c832-4f45-8cd7-e9759fe3f3f8-tmp\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.636879 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.636416    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4tm6\" (UniqueName: \"kubernetes.io/projected/66007fee-c832-4f45-8cd7-e9759fe3f3f8-kube-api-access-l4tm6\") pod \"tuned-lp6n8\" (UID: \"66007fee-c832-4f45-8cd7-e9759fe3f3f8\") " pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.636879 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.636454    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jksd\" (UniqueName: \"kubernetes.io/projected/5a25858a-ae63-4104-b1a4-33ac27e882df-kube-api-access-9jksd\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:10.636879 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.636774    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wmjp\" (UniqueName: \"kubernetes.io/projected/d917c83e-d627-4099-ba45-58674437b5a8-kube-api-access-5wmjp\") pod \"node-resolver-bhst2\" (UID: \"d917c83e-d627-4099-ba45-58674437b5a8\") " pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.636879 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.636851    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmd4j\" (UniqueName: \"kubernetes.io/projected/5b94e71e-bcd1-4b51-aeaa-c047a773ca50-kube-api-access-gmd4j\") pod \"multus-additional-cni-plugins-j7655\" (UID: \"5b94e71e-bcd1-4b51-aeaa-c047a773ca50\") " pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.637177 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.637099    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgl5m\" (UniqueName: \"kubernetes.io/projected/cc7866d7-9027-408c-a015-d30a286aede0-kube-api-access-bgl5m\") pod \"multus-28hw7\" (UID: \"cc7866d7-9027-408c-a015-d30a286aede0\") " pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.637177 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.637146    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kswwc\" (UniqueName: \"kubernetes.io/projected/f1b592e4-fbbf-4fe8-aa78-c10e97343d81-kube-api-access-kswwc\") pod \"node-ca-qbgl6\" (UID: \"f1b592e4-fbbf-4fe8-aa78-c10e97343d81\") " pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.728930 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.728896    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-kubelet\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.728930 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.728937    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-run-netns\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.728953    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-etc-openvswitch\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.728967    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-run-ovn\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.728983    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-sys-fs\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729004    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4vq8\" (UniqueName: \"kubernetes.io/projected/c9b36391-2125-47e3-80f5-4c880c52d2a3-kube-api-access-s4vq8\") pod \"iptables-alerter-hqmpp\" (UID: \"c9b36391-2125-47e3-80f5-4c880c52d2a3\") " pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729021    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-etc-openvswitch\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729031    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-run-netns\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729021    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-kubelet\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729028    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-run-openvswitch\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729076    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-sys-fs\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729038    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-run-ovn\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729095    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xksb4\" (UniqueName: \"kubernetes.io/projected/3a86f337-8566-420f-aab9-5d9329d8982c-kube-api-access-xksb4\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729127    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cee404ae-196d-4765-8dad-73a8c1da3793-agent-certs\") pod \"konnectivity-agent-z7jj4\" (UID: \"cee404ae-196d-4765-8dad-73a8c1da3793\") " pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729059    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-run-openvswitch\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729151    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b4434036-47bd-45c9-b08b-12ab9f5f84c6-kubelet-config\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:10.729168 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729175    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729222    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-systemd-units\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729248    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-var-lib-openvswitch\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729271    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b4434036-47bd-45c9-b08b-12ab9f5f84c6-kubelet-config\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729287    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-systemd-units\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.729280    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729281    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a86f337-8566-420f-aab9-5d9329d8982c-ovn-node-metrics-cert\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729315    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-var-lib-openvswitch\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.729360    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret podName:b4434036-47bd-45c9-b08b-12ab9f5f84c6 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:11.22933859 +0000 UTC m=+3.093588434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret") pod "global-pull-secret-syncer-sctjj" (UID: "b4434036-47bd-45c9-b08b-12ab9f5f84c6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729384    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729415    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3a86f337-8566-420f-aab9-5d9329d8982c-ovnkube-config\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729446    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-node-log\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729478    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-socket-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729505    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbf4w\" (UniqueName: \"kubernetes.io/projected/c224b479-8deb-459f-a0f9-565bf7aba451-kube-api-access-sbf4w\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729510    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-node-log\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729535    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729567    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-run-systemd\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.729897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729583    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729594    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-log-socket\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729617    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b4434036-47bd-45c9-b08b-12ab9f5f84c6-dbus\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729658    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-run-systemd\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729662    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-slash\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729689    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-slash\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729710    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-etc-selinux\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729717    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-log-socket\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729737    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729757    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b4434036-47bd-45c9-b08b-12ab9f5f84c6-dbus\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729790    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-registration-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729851    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-etc-selinux\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729859    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729907    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-registration-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.729907    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-device-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730007    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-device-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730010    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c9b36391-2125-47e3-80f5-4c880c52d2a3-iptables-alerter-script\") pod \"iptables-alerter-hqmpp\" (UID: \"c9b36391-2125-47e3-80f5-4c880c52d2a3\") " pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.730691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730066    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-cni-bin\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730083    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c224b479-8deb-459f-a0f9-565bf7aba451-socket-dir\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730092    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730128    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3a86f337-8566-420f-aab9-5d9329d8982c-env-overrides\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730131    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-cni-bin\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730156    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cee404ae-196d-4765-8dad-73a8c1da3793-konnectivity-ca\") pod \"konnectivity-agent-z7jj4\" (UID: \"cee404ae-196d-4765-8dad-73a8c1da3793\") " pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730186    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9b36391-2125-47e3-80f5-4c880c52d2a3-host-slash\") pod \"iptables-alerter-hqmpp\" (UID: \"c9b36391-2125-47e3-80f5-4c880c52d2a3\") " pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730211    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-cni-netd\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730212    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3a86f337-8566-420f-aab9-5d9329d8982c-ovnkube-config\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730237    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3a86f337-8566-420f-aab9-5d9329d8982c-ovnkube-script-lib\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730298    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9b36391-2125-47e3-80f5-4c880c52d2a3-host-slash\") pod \"iptables-alerter-hqmpp\" (UID: \"c9b36391-2125-47e3-80f5-4c880c52d2a3\") " pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730182    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730386    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3a86f337-8566-420f-aab9-5d9329d8982c-host-cni-netd\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730502    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c9b36391-2125-47e3-80f5-4c880c52d2a3-iptables-alerter-script\") pod \"iptables-alerter-hqmpp\" (UID: \"c9b36391-2125-47e3-80f5-4c880c52d2a3\") " pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730618    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3a86f337-8566-420f-aab9-5d9329d8982c-env-overrides\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730837    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cee404ae-196d-4765-8dad-73a8c1da3793-konnectivity-ca\") pod \"konnectivity-agent-z7jj4\" (UID: \"cee404ae-196d-4765-8dad-73a8c1da3793\") " pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:10.731291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.730986    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3a86f337-8566-420f-aab9-5d9329d8982c-ovnkube-script-lib\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.731939 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.731875    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a86f337-8566-420f-aab9-5d9329d8982c-ovn-node-metrics-cert\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.732000 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.731979    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cee404ae-196d-4765-8dad-73a8c1da3793-agent-certs\") pod \"konnectivity-agent-z7jj4\" (UID: \"cee404ae-196d-4765-8dad-73a8c1da3793\") " pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:10.736283 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.736248    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:30:10.736283 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.736275    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:30:10.736283 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.736289    2549 projected.go:196] Error preparing data for projected volume kube-api-access-rx42d for pod openshift-network-diagnostics/network-check-target-v7b58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:10.736553 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:10.736352    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d podName:0476c580-bf8f-4896-b103-fc129ad281b9 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:11.236337386 +0000 UTC m=+3.100587227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rx42d" (UniqueName: "kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d") pod "network-check-target-v7b58" (UID: "0476c580-bf8f-4896-b103-fc129ad281b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:10.738884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.738859    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4vq8\" (UniqueName: \"kubernetes.io/projected/c9b36391-2125-47e3-80f5-4c880c52d2a3-kube-api-access-s4vq8\") pod \"iptables-alerter-hqmpp\" (UID: \"c9b36391-2125-47e3-80f5-4c880c52d2a3\") " pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.739169 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.739148    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbf4w\" (UniqueName: \"kubernetes.io/projected/c224b479-8deb-459f-a0f9-565bf7aba451-kube-api-access-sbf4w\") pod \"aws-ebs-csi-driver-node-5965t\" (UID: \"c224b479-8deb-459f-a0f9-565bf7aba451\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:10.739270 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.739256    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xksb4\" (UniqueName: \"kubernetes.io/projected/3a86f337-8566-420f-aab9-5d9329d8982c-kube-api-access-xksb4\") pod \"ovnkube-node-7tctq\" (UID: \"3a86f337-8566-420f-aab9-5d9329d8982c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.826322 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.826286    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-28hw7"
Jul 28 12:30:10.834058 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.834022    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j7655"
Jul 28 12:30:10.841643 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.841608    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lp6n8"
Jul 28 12:30:10.846459 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.846413    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bhst2"
Jul 28 12:30:10.853336 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.853312    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qbgl6"
Jul 28 12:30:10.861343 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.861322    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hqmpp"
Jul 28 12:30:10.868179 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.868155    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:10.874146 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.874114    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:10.879601 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:10.879572    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t"
Jul 28 12:30:11.134440 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.134403    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:11.134667 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:11.134559    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:11.134667 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:11.134661    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:30:12.134619171 +0000 UTC m=+3.998869009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:11.235412 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.235368    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:11.235599 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:11.235557    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:11.235689 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:11.235659    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret podName:b4434036-47bd-45c9-b08b-12ab9f5f84c6 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:12.235619003 +0000 UTC m=+4.099868842 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret") pod "global-pull-secret-syncer-sctjj" (UID: "b4434036-47bd-45c9-b08b-12ab9f5f84c6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:11.336862 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.336770    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:11.337025 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:11.336930    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:30:11.337025 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:11.336949    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:30:11.337025 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:11.336958    2549 projected.go:196] Error preparing data for projected volume kube-api-access-rx42d for pod openshift-network-diagnostics/network-check-target-v7b58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:11.337025 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:11.337019    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d podName:0476c580-bf8f-4896-b103-fc129ad281b9 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:12.337000803 +0000 UTC m=+4.201250644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rx42d" (UniqueName: "kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d") pod "network-check-target-v7b58" (UID: "0476c580-bf8f-4896-b103-fc129ad281b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:11.504656 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:11.504610    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee404ae_196d_4765_8dad_73a8c1da3793.slice/crio-e6c220ad14648544a5f8543e6c3cbaa2a36e787aac28a2cad1b08c6bd0d09920 WatchSource:0}: Error finding container e6c220ad14648544a5f8543e6c3cbaa2a36e787aac28a2cad1b08c6bd0d09920: Status 404 returned error can't find the container with id e6c220ad14648544a5f8543e6c3cbaa2a36e787aac28a2cad1b08c6bd0d09920
Jul 28 12:30:11.512519 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:11.512492    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66007fee_c832_4f45_8cd7_e9759fe3f3f8.slice/crio-2c19776459c95c4b759b012e944426335e5b6658c329eeb4ad760b0f0b3e5fac WatchSource:0}: Error finding container 2c19776459c95c4b759b012e944426335e5b6658c329eeb4ad760b0f0b3e5fac: Status 404 returned error can't find the container with id 2c19776459c95c4b759b012e944426335e5b6658c329eeb4ad760b0f0b3e5fac
Jul 28 12:30:11.514060 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:11.514029    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc7866d7_9027_408c_a015_d30a286aede0.slice/crio-3643d3d26e5974d0d478e9784cc5ed91a1daadff7f2d03397d1de570801450b2 WatchSource:0}: Error finding container 3643d3d26e5974d0d478e9784cc5ed91a1daadff7f2d03397d1de570801450b2: Status 404 returned error can't find the container with id 3643d3d26e5974d0d478e9784cc5ed91a1daadff7f2d03397d1de570801450b2
Jul 28 12:30:11.515493 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:11.514873    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc224b479_8deb_459f_a0f9_565bf7aba451.slice/crio-3656dfbf4109d1a2a330e61437535ec0cb885aad72e6ffc6e7389e62152c6ca7 WatchSource:0}: Error finding container 3656dfbf4109d1a2a330e61437535ec0cb885aad72e6ffc6e7389e62152c6ca7: Status 404 returned error can't find the container with id 3656dfbf4109d1a2a330e61437535ec0cb885aad72e6ffc6e7389e62152c6ca7
Jul 28 12:30:11.517220 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:11.517119    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd917c83e_d627_4099_ba45_58674437b5a8.slice/crio-d74b5a293f0b5e29e1f5ffa93cbcb1e31a719cb8b56ccd91c5373302863427b3 WatchSource:0}: Error finding container d74b5a293f0b5e29e1f5ffa93cbcb1e31a719cb8b56ccd91c5373302863427b3: Status 404 returned error can't find the container with id d74b5a293f0b5e29e1f5ffa93cbcb1e31a719cb8b56ccd91c5373302863427b3
Jul 28 12:30:11.517797 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:11.517774    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b94e71e_bcd1_4b51_aeaa_c047a773ca50.slice/crio-1e74d2d0d26b44ff16d152ac8d5ec42e598d7cc7be600700120d346fbe500016 WatchSource:0}: Error finding container 1e74d2d0d26b44ff16d152ac8d5ec42e598d7cc7be600700120d346fbe500016: Status 404 returned error can't find the container with id 1e74d2d0d26b44ff16d152ac8d5ec42e598d7cc7be600700120d346fbe500016
Jul 28 12:30:11.519220 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:11.519196    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b592e4_fbbf_4fe8_aa78_c10e97343d81.slice/crio-5e4b9a5a7336740fdb19ca2385beb12d6a72dfbf153adc7829052d23a080b542 WatchSource:0}: Error finding container 5e4b9a5a7336740fdb19ca2385beb12d6a72dfbf153adc7829052d23a080b542: Status 404 returned error can't find the container with id 5e4b9a5a7336740fdb19ca2385beb12d6a72dfbf153adc7829052d23a080b542
Jul 28 12:30:11.520731 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:11.520672    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b36391_2125_47e3_80f5_4c880c52d2a3.slice/crio-f38e53bb0ec9d90758da3e45cd132909dde6d6d2b3aedd19626c711797eeb87a WatchSource:0}: Error finding container f38e53bb0ec9d90758da3e45cd132909dde6d6d2b3aedd19626c711797eeb87a: Status 404 returned error can't find the container with id f38e53bb0ec9d90758da3e45cd132909dde6d6d2b3aedd19626c711797eeb87a
Jul 28 12:30:11.568753 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.568701    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 12:25:10 +0000 UTC" deadline="2028-02-13 02:03:50.334993161 +0000 UTC"
Jul 28 12:30:11.568753 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.568747    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13549h33m38.766249398s"
Jul 28 12:30:11.695269 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.695057    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerStarted","Data":"91eabb4f87567d9da843594a0b320c9b0ae364488767fdb88f1fe8f969ab420f"}
Jul 28 12:30:11.696230 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.696201    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7655" event={"ID":"5b94e71e-bcd1-4b51-aeaa-c047a773ca50","Type":"ContainerStarted","Data":"1e74d2d0d26b44ff16d152ac8d5ec42e598d7cc7be600700120d346fbe500016"}
Jul 28 12:30:11.698047 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.698019    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal" event={"ID":"9415a092b6cee87f2b0f814e0eb8bce0","Type":"ContainerStarted","Data":"0e96464fc30ca9e5df3292bd47f7c6e9030b7749323e01ea0218e3e1ffed2760"}
Jul 28 12:30:11.699150 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.699127    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hqmpp" event={"ID":"c9b36391-2125-47e3-80f5-4c880c52d2a3","Type":"ContainerStarted","Data":"f38e53bb0ec9d90758da3e45cd132909dde6d6d2b3aedd19626c711797eeb87a"}
Jul 28 12:30:11.700240 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.700214    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bhst2" event={"ID":"d917c83e-d627-4099-ba45-58674437b5a8","Type":"ContainerStarted","Data":"d74b5a293f0b5e29e1f5ffa93cbcb1e31a719cb8b56ccd91c5373302863427b3"}
Jul 28 12:30:11.701491 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.701464    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-28hw7" event={"ID":"cc7866d7-9027-408c-a015-d30a286aede0","Type":"ContainerStarted","Data":"3643d3d26e5974d0d478e9784cc5ed91a1daadff7f2d03397d1de570801450b2"}
Jul 28 12:30:11.702604 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.702585    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lp6n8" event={"ID":"66007fee-c832-4f45-8cd7-e9759fe3f3f8","Type":"ContainerStarted","Data":"2c19776459c95c4b759b012e944426335e5b6658c329eeb4ad760b0f0b3e5fac"}
Jul 28 12:30:11.703568 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.703550    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z7jj4" event={"ID":"cee404ae-196d-4765-8dad-73a8c1da3793","Type":"ContainerStarted","Data":"e6c220ad14648544a5f8543e6c3cbaa2a36e787aac28a2cad1b08c6bd0d09920"}
Jul 28 12:30:11.704668 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.704648    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qbgl6" event={"ID":"f1b592e4-fbbf-4fe8-aa78-c10e97343d81","Type":"ContainerStarted","Data":"5e4b9a5a7336740fdb19ca2385beb12d6a72dfbf153adc7829052d23a080b542"}
Jul 28 12:30:11.705562 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.705543    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t" event={"ID":"c224b479-8deb-459f-a0f9-565bf7aba451","Type":"ContainerStarted","Data":"3656dfbf4109d1a2a330e61437535ec0cb885aad72e6ffc6e7389e62152c6ca7"}
Jul 28 12:30:11.713936 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:11.713896    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-90.ec2.internal" podStartSLOduration=1.7138827399999998 podStartE2EDuration="1.71388274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:11.713692899 +0000 UTC m=+3.577942759" watchObservedRunningTime="2026-07-28 12:30:11.71388274 +0000 UTC m=+3.578132599"
Jul 28 12:30:12.144911 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.144870    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:12.145101 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.145031    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:12.145101 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.145094    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:30:14.145074705 +0000 UTC m=+6.009324546 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:12.245694 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.245651    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:12.245884 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.245865    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:12.245951 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.245938    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret podName:b4434036-47bd-45c9-b08b-12ab9f5f84c6 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:14.245917306 +0000 UTC m=+6.110167146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret") pod "global-pull-secret-syncer-sctjj" (UID: "b4434036-47bd-45c9-b08b-12ab9f5f84c6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:12.346381 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.346340    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:12.346615 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.346594    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:30:12.346730 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.346644    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:30:12.346730 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.346660    2549 projected.go:196] Error preparing data for projected volume kube-api-access-rx42d for pod openshift-network-diagnostics/network-check-target-v7b58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:12.346730 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.346722    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d podName:0476c580-bf8f-4896-b103-fc129ad281b9 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:14.34670281 +0000 UTC m=+6.210952672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rx42d" (UniqueName: "kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d") pod "network-check-target-v7b58" (UID: "0476c580-bf8f-4896-b103-fc129ad281b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:12.569438 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.569302    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 12:25:10 +0000 UTC" deadline="2028-04-22 20:51:00.306749802 +0000 UTC"
Jul 28 12:30:12.569438 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.569342    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15224h20m47.737411864s"
Jul 28 12:30:12.688941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.688909    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:12.689113 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.689043    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:12.689472 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.689451    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:12.689559 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.689546    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:12.689795 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.689618    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:12.689795 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:12.689727    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:12.716533 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.716490    2549 generic.go:356] "Generic (PLEG): container finished" podID="3ec3e761d1583b4f7566737026b3db1f" containerID="7d18ab94b62f691cf629d75936abe01c4c4024fef4794edd6935ddb439693f39" exitCode=0
Jul 28 12:30:12.716718 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:12.716670    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal" event={"ID":"3ec3e761d1583b4f7566737026b3db1f","Type":"ContainerDied","Data":"7d18ab94b62f691cf629d75936abe01c4c4024fef4794edd6935ddb439693f39"}
Jul 28 12:30:13.732508 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:13.732467    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal" event={"ID":"3ec3e761d1583b4f7566737026b3db1f","Type":"ContainerStarted","Data":"ec25cf1a3aa8c21e24e4c446ce8882ff25312de9f35ac1628f7b78c02feccc95"}
Jul 28 12:30:14.161799 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:14.161596    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:14.161970 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.161843    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:14.161970 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.161933    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:30:18.161907217 +0000 UTC m=+10.026157058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:14.262735 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:14.262696    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:14.262960 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.262936    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:14.263071 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.263009    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret podName:b4434036-47bd-45c9-b08b-12ab9f5f84c6 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:18.262991569 +0000 UTC m=+10.127241414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret") pod "global-pull-secret-syncer-sctjj" (UID: "b4434036-47bd-45c9-b08b-12ab9f5f84c6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:14.363817 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:14.363774    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:14.363994 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.363962    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:30:14.363994 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.363981    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:30:14.363994 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.363994    2549 projected.go:196] Error preparing data for projected volume kube-api-access-rx42d for pod openshift-network-diagnostics/network-check-target-v7b58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:14.364145 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.364060    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d podName:0476c580-bf8f-4896-b103-fc129ad281b9 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:18.364039754 +0000 UTC m=+10.228289595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rx42d" (UniqueName: "kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d") pod "network-check-target-v7b58" (UID: "0476c580-bf8f-4896-b103-fc129ad281b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:14.687304 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:14.686818    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:14.687304 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.686975    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:14.687304 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:14.686994    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:14.687304 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.687103    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:14.687304 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:14.687161    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:14.687304 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:14.687232    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:16.687744 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:16.687154    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:16.687744 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:16.687195    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:16.687744 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:16.687305    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:16.687744 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:16.687351    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:16.687744 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:16.687481    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:16.687744 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:16.687557    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:18.197811 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:18.197759    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:18.198248 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.197966    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:18.198248 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.198037    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:30:26.198015564 +0000 UTC m=+18.062265403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:18.298799 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:18.298752    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:18.298971 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.298949    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:18.299025 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.299012    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret podName:b4434036-47bd-45c9-b08b-12ab9f5f84c6 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:26.298995194 +0000 UTC m=+18.163245036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret") pod "global-pull-secret-syncer-sctjj" (UID: "b4434036-47bd-45c9-b08b-12ab9f5f84c6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:18.399452 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:18.399412    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:18.399648 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.399617    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:30:18.399723 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.399655    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:30:18.399723 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.399668    2549 projected.go:196] Error preparing data for projected volume kube-api-access-rx42d for pod openshift-network-diagnostics/network-check-target-v7b58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:18.399723 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.399722    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d podName:0476c580-bf8f-4896-b103-fc129ad281b9 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:26.399704932 +0000 UTC m=+18.263954768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rx42d" (UniqueName: "kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d") pod "network-check-target-v7b58" (UID: "0476c580-bf8f-4896-b103-fc129ad281b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:18.687226 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:18.687144    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:18.687226 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:18.687178    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:18.687226 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:18.687157    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:18.687480 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.687272    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:18.687480 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.687336    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:18.687480 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:18.687404    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:20.689590 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:20.688737    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:20.689590 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:20.688873    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:20.689590 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:20.689293    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:20.689590 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:20.689386    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:20.689590 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:20.689452    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:20.689590 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:20.689537    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:22.688819 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:22.688788    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:22.688819 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:22.688820    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:22.689306 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:22.688797    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:22.689306 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:22.688903    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:22.689306 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:22.688993    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:22.689306 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:22.689114    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:24.686643 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:24.686592    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:24.687169 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:24.686583    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:24.687169 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:24.686756    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:24.687169 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:24.686738    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:24.687169 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:24.686872    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:24.687169 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:24.686943    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:26.263784 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:26.263749    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:26.264210 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.263888    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:26.264210 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.263950    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:30:42.263932948 +0000 UTC m=+34.128182784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:26.364988 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:26.364948    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:26.365145 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.365126    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:26.365259 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.365204    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret podName:b4434036-47bd-45c9-b08b-12ab9f5f84c6 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:42.365183891 +0000 UTC m=+34.229433733 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret") pod "global-pull-secret-syncer-sctjj" (UID: "b4434036-47bd-45c9-b08b-12ab9f5f84c6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:26.466270 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:26.466225    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:26.466424 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.466402    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:30:26.466463 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.466427    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:30:26.466463 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.466437    2549 projected.go:196] Error preparing data for projected volume kube-api-access-rx42d for pod openshift-network-diagnostics/network-check-target-v7b58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:26.466561 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.466486    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d podName:0476c580-bf8f-4896-b103-fc129ad281b9 nodeName:}" failed. No retries permitted until 2026-07-28 12:30:42.466473764 +0000 UTC m=+34.330723602 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rx42d" (UniqueName: "kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d") pod "network-check-target-v7b58" (UID: "0476c580-bf8f-4896-b103-fc129ad281b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:26.686559 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:26.686465    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:26.686740 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:26.686465    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:26.686740 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:26.686591    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:26.686740 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.686615    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:26.686740 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.686689    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:26.686959 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:26.686756    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:28.687050 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:28.687022    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:28.687556 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:28.687118    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:28.687556 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:28.687178    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:28.687556 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:28.687231    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:28.687556 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:28.687284    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:28.687556 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:28.687352    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:30.685893 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.685656    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:30.686706 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:30.685970    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:30.686706 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.685691    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:30.686706 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:30.686063    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:30.686706 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.685658    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:30.686706 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:30.686137    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:30.769156 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.769130    2549 generic.go:356] "Generic (PLEG): container finished" podID="5b94e71e-bcd1-4b51-aeaa-c047a773ca50" containerID="1e80235c7f66f477da33ad7b9320adcdc1311d92da9e5ffeede5c490467a7c10" exitCode=0
Jul 28 12:30:30.769292 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.769213    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7655" event={"ID":"5b94e71e-bcd1-4b51-aeaa-c047a773ca50","Type":"ContainerDied","Data":"1e80235c7f66f477da33ad7b9320adcdc1311d92da9e5ffeede5c490467a7c10"}
Jul 28 12:30:30.770780 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.770756    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bhst2" event={"ID":"d917c83e-d627-4099-ba45-58674437b5a8","Type":"ContainerStarted","Data":"c948f67ba0d2ce2038efd6a0d2ae939a090ccf4689fd5fc6e1fbec74e529aa2a"}
Jul 28 12:30:30.772306 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.772274    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-28hw7" event={"ID":"cc7866d7-9027-408c-a015-d30a286aede0","Type":"ContainerStarted","Data":"c09e20364c6498b9137717ad7d12d939b015bb9f84bdcef2bfad2f0ec72c695d"}
Jul 28 12:30:30.773951 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.773916    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lp6n8" event={"ID":"66007fee-c832-4f45-8cd7-e9759fe3f3f8","Type":"ContainerStarted","Data":"6ba6c972b0ebaf894bc404a5e8cfed2361425713d57776ae518bc6bc543c6e12"}
Jul 28 12:30:30.775466 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.775441    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z7jj4" event={"ID":"cee404ae-196d-4765-8dad-73a8c1da3793","Type":"ContainerStarted","Data":"c80b136bbd9b4d59abf6c48e997433706751de17b7a2fd652e52bde60081bfbe"}
Jul 28 12:30:30.776925 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.776904    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qbgl6" event={"ID":"f1b592e4-fbbf-4fe8-aa78-c10e97343d81","Type":"ContainerStarted","Data":"baa89a797b5285a1c5e388349abe867b9090c5de8e85cd59b184abdfc51b13d7"}
Jul 28 12:30:30.778263 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.778238    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t" event={"ID":"c224b479-8deb-459f-a0f9-565bf7aba451","Type":"ContainerStarted","Data":"f5608d815dea8cf584dc365c99d4a2bff898879a8dcce78ad9542995d818261f"}
Jul 28 12:30:30.780559 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.780540    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:30:30.780939 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.780918    2549 generic.go:356] "Generic (PLEG): container finished" podID="3a86f337-8566-420f-aab9-5d9329d8982c" containerID="20d0ccc4d4bd78af8e8835b1fe28e259a8454225ab6419408472f5fba014cc69" exitCode=1
Jul 28 12:30:30.781000 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.780954    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerStarted","Data":"b080bcc04b926fbb62ddf05d3e46c1367cf56af36f8e970fbc09b1ee5b55db43"}
Jul 28 12:30:30.781062 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.781047    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerStarted","Data":"96d3114b026fd963c135b3f29ef93e476a8141a0850c2733cd3060940c56a3b1"}
Jul 28 12:30:30.781113 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.781099    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerDied","Data":"20d0ccc4d4bd78af8e8835b1fe28e259a8454225ab6419408472f5fba014cc69"}
Jul 28 12:30:30.781167 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.781154    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerStarted","Data":"c6e99bce4a9253fa13610c97df6191e086d8b4c3d40a0661f07632bb5b6e7233"}
Jul 28 12:30:30.796818 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.796778    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-90.ec2.internal" podStartSLOduration=20.79676303 podStartE2EDuration="20.79676303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:13.749035843 +0000 UTC m=+5.613285716" watchObservedRunningTime="2026-07-28 12:30:30.79676303 +0000 UTC m=+22.661012888"
Jul 28 12:30:30.842579 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.842526    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lp6n8" podStartSLOduration=4.2578369 podStartE2EDuration="22.842509807s" totalImagesPullingTime="18.584672907s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:30.84207877 +0000 UTC m=+22.706328629" watchObservedRunningTime="2026-07-28 12:30:30.842509807 +0000 UTC m=+22.706759667"
Jul 28 12:30:30.842770 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.842740    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-28hw7" podStartSLOduration=4.231621348 podStartE2EDuration="22.842732411s" totalImagesPullingTime="18.611111063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:30.816992169 +0000 UTC m=+22.681242028" watchObservedRunningTime="2026-07-28 12:30:30.842732411 +0000 UTC m=+22.706982271"
Jul 28 12:30:30.863221 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.863154    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bhst2" podStartSLOduration=4.312652879 podStartE2EDuration="22.863139253s" totalImagesPullingTime="18.550486374s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:30.863066331 +0000 UTC m=+22.727316191" watchObservedRunningTime="2026-07-28 12:30:30.863139253 +0000 UTC m=+22.727389111"
Jul 28 12:30:30.888032 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.887874    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qbgl6" podStartSLOduration=4.2937602009999996 podStartE2EDuration="22.88785482s" totalImagesPullingTime="18.594094619s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:30.887007137 +0000 UTC m=+22.751256998" watchObservedRunningTime="2026-07-28 12:30:30.88785482 +0000 UTC m=+22.752104680"
Jul 28 12:30:30.904933 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:30.904876    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z7jj4" podStartSLOduration=4.341784741 podStartE2EDuration="22.904855485s" totalImagesPullingTime="18.563070744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:30.904798759 +0000 UTC m=+22.769048619" watchObservedRunningTime="2026-07-28 12:30:30.904855485 +0000 UTC m=+22.769105345"
Jul 28 12:30:31.506661 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:31.506607    2549 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 12:30:31.594265 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:31.594143    2549 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T12:30:31.506654751Z","UUID":"61165f43-7dad-4457-acbb-53976d2b460d","Handler":null,"Name":"","Endpoint":""}
Jul 28 12:30:31.595949 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:31.595927    2549 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 12:30:31.596064 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:31.595978    2549 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 12:30:31.785770 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:31.785733    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hqmpp" event={"ID":"c9b36391-2125-47e3-80f5-4c880c52d2a3","Type":"ContainerStarted","Data":"0af114f843ff8526aec7e4878a7b6c50557034b19beede09293353c9baccc7f6"}
Jul 28 12:30:31.788415 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:31.788382    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t" event={"ID":"c224b479-8deb-459f-a0f9-565bf7aba451","Type":"ContainerStarted","Data":"5e99add1b4818a6b256b508fcb20aa02a6d90e6459e535d983c1696492c39a43"}
Jul 28 12:30:31.791337 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:31.791306    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:30:31.791710 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:31.791683    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerStarted","Data":"3eff6fd3257019d7f3c2125758b2f0ffd516df253379fc4bba989862a58268ac"}
Jul 28 12:30:31.791816 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:31.791771    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerStarted","Data":"e8cf9c69be47e8287e5381294e18b5c669c7e3d291e04e59914fa0134c1d2160"}
Jul 28 12:30:32.688673 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:32.688640    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:32.688842 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:32.688688    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:32.688842 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:32.688770    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:32.688842 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:32.688782    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:32.688971 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:32.688879    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:32.689010 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:32.688986    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:33.799568 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:33.799525    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t" event={"ID":"c224b479-8deb-459f-a0f9-565bf7aba451","Type":"ContainerStarted","Data":"178ae02ed2abc3bafa07d0020db30cec82e0480ef3cb5ad71df5a70afdd9a221"}
Jul 28 12:30:33.803050 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:33.803024    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:30:33.803471 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:33.803431    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerStarted","Data":"2994583eefa7ba299ac328fbb775d11eab27db3b6ebd4e8f924c6ac55b44eb91"}
Jul 28 12:30:33.819716 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:33.819655    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hqmpp" podStartSLOduration=7.222333992 podStartE2EDuration="25.81961143s" totalImagesPullingTime="18.597277438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:31.824295823 +0000 UTC m=+23.688545687" watchObservedRunningTime="2026-07-28 12:30:33.81961143 +0000 UTC m=+25.683861292"
Jul 28 12:30:33.819905 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:33.819811    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5965t" podStartSLOduration=5.002431864 podStartE2EDuration="25.819802044s" totalImagesPullingTime="20.81737018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:33.817684231 +0000 UTC m=+25.681934090" watchObservedRunningTime="2026-07-28 12:30:33.819802044 +0000 UTC m=+25.684051904"
Jul 28 12:30:34.686577 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:34.686389    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:34.686747 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:34.686389    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:34.686747 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:34.686693    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:34.686747 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:34.686428    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:34.686889 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:34.686765    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:34.686889 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:34.686845    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:34.911923 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:34.911884    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:34.912511 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:34.912495    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:35.811601 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.811522    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:30:35.811968 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.811944    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerStarted","Data":"9f2085db94a379f7d166cd7532356b83d34b3f19fd43dd54799decaa1fe35efb"}
Jul 28 12:30:35.812271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.812238    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:35.812360 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.812320    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:35.812454 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.812435    2549 scope.go:117] "RemoveContainer" containerID="20d0ccc4d4bd78af8e8835b1fe28e259a8454225ab6419408472f5fba014cc69"
Jul 28 12:30:35.814053 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.814013    2549 generic.go:356] "Generic (PLEG): container finished" podID="5b94e71e-bcd1-4b51-aeaa-c047a773ca50" containerID="07aac52e603b75b6a24ef7ca608a31f92562b3b09f6a70a0a82ddb03bced3fbc" exitCode=0
Jul 28 12:30:35.814196 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.814095    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7655" event={"ID":"5b94e71e-bcd1-4b51-aeaa-c047a773ca50","Type":"ContainerDied","Data":"07aac52e603b75b6a24ef7ca608a31f92562b3b09f6a70a0a82ddb03bced3fbc"}
Jul 28 12:30:35.814363 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.814347    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:35.815066 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.815050    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z7jj4"
Jul 28 12:30:35.830557 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:35.830532    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:36.686039 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:36.685844    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:36.686884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:36.685844    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:36.686884 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:36.686143    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:36.686884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:36.685905    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:36.686884 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:36.686213    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:36.686884 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:36.686282    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:36.822322 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:36.822292    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:30:36.822680 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:36.822650    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" event={"ID":"3a86f337-8566-420f-aab9-5d9329d8982c","Type":"ContainerStarted","Data":"8abe4fef55b6e336a1b461c8f27c50fafd137214af244c2fd06e9714a000d2a9"}
Jul 28 12:30:36.822817 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:36.822799    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:36.839230 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:36.839200    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:30:36.854846 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:36.854797    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq" podStartSLOduration=10.217576863 podStartE2EDuration="28.854782284s" totalImagesPullingTime="18.637205421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:36.853245893 +0000 UTC m=+28.717495751" watchObservedRunningTime="2026-07-28 12:30:36.854782284 +0000 UTC m=+28.719032143"
Jul 28 12:30:37.413356 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:37.413286    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sctjj"]
Jul 28 12:30:37.413556 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:37.413442    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:37.413556 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:37.413527    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:37.416827 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:37.416801    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mg9wl"]
Jul 28 12:30:37.416982 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:37.416907    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:37.417043 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:37.417021    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:37.417404 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:37.417372    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v7b58"]
Jul 28 12:30:37.417506 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:37.417465    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:37.417561 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:37.417547    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:37.827351 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:37.827317    2549 generic.go:356] "Generic (PLEG): container finished" podID="5b94e71e-bcd1-4b51-aeaa-c047a773ca50" containerID="003ec282b6090c1511e45319ffa9b1a85d8f1059ba0b56654cff99d8f124b9dc" exitCode=0
Jul 28 12:30:37.827912 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:37.827398    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7655" event={"ID":"5b94e71e-bcd1-4b51-aeaa-c047a773ca50","Type":"ContainerDied","Data":"003ec282b6090c1511e45319ffa9b1a85d8f1059ba0b56654cff99d8f124b9dc"}
Jul 28 12:30:38.686954 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:38.686924    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:38.687070 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:38.687047    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:38.687124 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:38.687091    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:38.687212 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:38.687190    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:38.832721 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:38.832611    2549 generic.go:356] "Generic (PLEG): container finished" podID="5b94e71e-bcd1-4b51-aeaa-c047a773ca50" containerID="fba41775fce4338672711096be8f6339d2328b32ea88c6dd773bb5b0ad55aa6b" exitCode=0
Jul 28 12:30:38.832721 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:38.832665    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7655" event={"ID":"5b94e71e-bcd1-4b51-aeaa-c047a773ca50","Type":"ContainerDied","Data":"fba41775fce4338672711096be8f6339d2328b32ea88c6dd773bb5b0ad55aa6b"}
Jul 28 12:30:39.685614 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:39.685585    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:39.685794 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:39.685711    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:40.685538 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:40.685508    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:40.686571 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:40.685508    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:40.686571 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:40.685674    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:40.686571 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:40.685716    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:41.686241 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:41.686193    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:41.686705 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:41.686355    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sctjj" podUID="b4434036-47bd-45c9-b08b-12ab9f5f84c6"
Jul 28 12:30:42.286922 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.286885    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:42.287188 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.287079    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:42.287188 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.287167    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:31:14.287145289 +0000 UTC m=+66.151395127 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 12:30:42.387270 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.387231    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:42.387469 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.387357    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:42.387469 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.387427    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret podName:b4434036-47bd-45c9-b08b-12ab9f5f84c6 nodeName:}" failed. No retries permitted until 2026-07-28 12:31:14.387410251 +0000 UTC m=+66.251660111 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret") pod "global-pull-secret-syncer-sctjj" (UID: "b4434036-47bd-45c9-b08b-12ab9f5f84c6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 12:30:42.487832 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.487794    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:42.488023 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.487988    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 12:30:42.488023 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.488014    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 12:30:42.488135 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.488026    2549 projected.go:196] Error preparing data for projected volume kube-api-access-rx42d for pod openshift-network-diagnostics/network-check-target-v7b58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:42.488135 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.488088    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d podName:0476c580-bf8f-4896-b103-fc129ad281b9 nodeName:}" failed. No retries permitted until 2026-07-28 12:31:14.488073988 +0000 UTC m=+66.352323825 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rx42d" (UniqueName: "kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d") pod "network-check-target-v7b58" (UID: "0476c580-bf8f-4896-b103-fc129ad281b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 12:30:42.686104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.686066    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:42.686284 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.686195    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7b58" podUID="0476c580-bf8f-4896-b103-fc129ad281b9"
Jul 28 12:30:42.686284 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.686240    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:42.686682 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:42.686348    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:30:42.909398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.909367    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-90.ec2.internal" event="NodeReady"
Jul 28 12:30:42.909580 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.909517    2549 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 12:30:42.955402 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.955321    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qc9cp"]
Jul 28 12:30:42.968272 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.968219    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hgnwd"]
Jul 28 12:30:42.968446 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.968395    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:42.971618 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.971595    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bstgx\""
Jul 28 12:30:42.971859 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.971844    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 12:30:42.972082 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.972065    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 12:30:42.990409 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.990378    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qc9cp"]
Jul 28 12:30:42.990655 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.990511    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hgnwd"
Jul 28 12:30:42.990655 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.990555    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hgnwd"]
Jul 28 12:30:42.993892 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.993872    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 12:30:42.994019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.993919    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-74b58\""
Jul 28 12:30:42.994019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.993872    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 12:30:42.994019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:42.993874    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 12:30:43.091494 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.091451    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.091712 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.091532    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8168db1d-be3a-4555-9a2d-0876f4a02c1a-config-volume\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.091712 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.091567    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cxhf\" (UniqueName: \"kubernetes.io/projected/12e4bb44-c3f8-48b5-bc8c-a43380db0819-kube-api-access-5cxhf\") pod \"ingress-canary-hgnwd\" (UID: \"12e4bb44-c3f8-48b5-bc8c-a43380db0819\") " pod="openshift-ingress-canary/ingress-canary-hgnwd"
Jul 28 12:30:43.091712 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.091606    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12e4bb44-c3f8-48b5-bc8c-a43380db0819-cert\") pod \"ingress-canary-hgnwd\" (UID: \"12e4bb44-c3f8-48b5-bc8c-a43380db0819\") " pod="openshift-ingress-canary/ingress-canary-hgnwd"
Jul 28 12:30:43.091712 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.091656    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbcdc\" (UniqueName: \"kubernetes.io/projected/8168db1d-be3a-4555-9a2d-0876f4a02c1a-kube-api-access-sbcdc\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.091712 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.091687    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8168db1d-be3a-4555-9a2d-0876f4a02c1a-tmp-dir\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.192788 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.192742    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12e4bb44-c3f8-48b5-bc8c-a43380db0819-cert\") pod \"ingress-canary-hgnwd\" (UID: \"12e4bb44-c3f8-48b5-bc8c-a43380db0819\") " pod="openshift-ingress-canary/ingress-canary-hgnwd"
Jul 28 12:30:43.192983 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.192799    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbcdc\" (UniqueName: \"kubernetes.io/projected/8168db1d-be3a-4555-9a2d-0876f4a02c1a-kube-api-access-sbcdc\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.192983 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.192829    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8168db1d-be3a-4555-9a2d-0876f4a02c1a-tmp-dir\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.192983 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.192881    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.192983 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:43.192975    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:30:43.193187 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.192984    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8168db1d-be3a-4555-9a2d-0876f4a02c1a-config-volume\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.193187 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.193027    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cxhf\" (UniqueName: \"kubernetes.io/projected/12e4bb44-c3f8-48b5-bc8c-a43380db0819-kube-api-access-5cxhf\") pod \"ingress-canary-hgnwd\" (UID: \"12e4bb44-c3f8-48b5-bc8c-a43380db0819\") " pod="openshift-ingress-canary/ingress-canary-hgnwd"
Jul 28 12:30:43.193187 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:43.193056    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls podName:8168db1d-be3a-4555-9a2d-0876f4a02c1a nodeName:}" failed. No retries permitted until 2026-07-28 12:30:43.693035916 +0000 UTC m=+35.557285776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls") pod "dns-default-qc9cp" (UID: "8168db1d-be3a-4555-9a2d-0876f4a02c1a") : secret "dns-default-metrics-tls" not found
Jul 28 12:30:43.193316 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.193214    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8168db1d-be3a-4555-9a2d-0876f4a02c1a-tmp-dir\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.193438 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.193417    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8168db1d-be3a-4555-9a2d-0876f4a02c1a-config-volume\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.197414 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.197383    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12e4bb44-c3f8-48b5-bc8c-a43380db0819-cert\") pod \"ingress-canary-hgnwd\" (UID: \"12e4bb44-c3f8-48b5-bc8c-a43380db0819\") " pod="openshift-ingress-canary/ingress-canary-hgnwd"
Jul 28 12:30:43.204946 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.204913    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cxhf\" (UniqueName: \"kubernetes.io/projected/12e4bb44-c3f8-48b5-bc8c-a43380db0819-kube-api-access-5cxhf\") pod \"ingress-canary-hgnwd\" (UID: \"12e4bb44-c3f8-48b5-bc8c-a43380db0819\") " pod="openshift-ingress-canary/ingress-canary-hgnwd"
Jul 28 12:30:43.206744 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.206682    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbcdc\" (UniqueName: \"kubernetes.io/projected/8168db1d-be3a-4555-9a2d-0876f4a02c1a-kube-api-access-sbcdc\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.299488 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.299443    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hgnwd"
Jul 28 12:30:43.517164 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.514751    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hgnwd"]
Jul 28 12:30:43.519648 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:30:43.519595    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e4bb44_c3f8_48b5_bc8c_a43380db0819.slice/crio-00d1e55e2e3b0741efe1e1d20c96305b0644a811e54af2223ed6965c927fd0f6 WatchSource:0}: Error finding container 00d1e55e2e3b0741efe1e1d20c96305b0644a811e54af2223ed6965c927fd0f6: Status 404 returned error can't find the container with id 00d1e55e2e3b0741efe1e1d20c96305b0644a811e54af2223ed6965c927fd0f6
Jul 28 12:30:43.685893 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.685854    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:30:43.689086 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.689066    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 12:30:43.697352 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.697328    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:43.697497 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:43.697479    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:30:43.697549 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:43.697539    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls podName:8168db1d-be3a-4555-9a2d-0876f4a02c1a nodeName:}" failed. No retries permitted until 2026-07-28 12:30:44.697524636 +0000 UTC m=+36.561774475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls") pod "dns-default-qc9cp" (UID: "8168db1d-be3a-4555-9a2d-0876f4a02c1a") : secret "dns-default-metrics-tls" not found
Jul 28 12:30:43.846933 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.846897    2549 generic.go:356] "Generic (PLEG): container finished" podID="5b94e71e-bcd1-4b51-aeaa-c047a773ca50" containerID="615c404150bc435ac90a0006d2e82cc1aa120145cb288454c42f8168d63f9c9f" exitCode=0
Jul 28 12:30:43.847109 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.846990    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7655" event={"ID":"5b94e71e-bcd1-4b51-aeaa-c047a773ca50","Type":"ContainerDied","Data":"615c404150bc435ac90a0006d2e82cc1aa120145cb288454c42f8168d63f9c9f"}
Jul 28 12:30:43.848226 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:43.848154    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hgnwd" event={"ID":"12e4bb44-c3f8-48b5-bc8c-a43380db0819","Type":"ContainerStarted","Data":"00d1e55e2e3b0741efe1e1d20c96305b0644a811e54af2223ed6965c927fd0f6"}
Jul 28 12:30:44.686504 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.686469    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:30:44.686716 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.686476    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:30:44.689889 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.689862    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 12:30:44.690273 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.689869    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 12:30:44.691288 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.691266    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-klsqn\""
Jul 28 12:30:44.691414 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.691347    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 12:30:44.691414 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.691270    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nm4t9\""
Jul 28 12:30:44.705959 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.705927    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:44.706114 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:44.706094    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:30:44.706184 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:44.706172    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls podName:8168db1d-be3a-4555-9a2d-0876f4a02c1a nodeName:}" failed. No retries permitted until 2026-07-28 12:30:46.706150578 +0000 UTC m=+38.570400422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls") pod "dns-default-qc9cp" (UID: "8168db1d-be3a-4555-9a2d-0876f4a02c1a") : secret "dns-default-metrics-tls" not found
Jul 28 12:30:44.853859 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.853798    2549 generic.go:356] "Generic (PLEG): container finished" podID="5b94e71e-bcd1-4b51-aeaa-c047a773ca50" containerID="c636faf0d78538d1b823c545125fa1151a56cfcd32d1f1f81b099c095078d229" exitCode=0
Jul 28 12:30:44.853859 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:44.853858    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7655" event={"ID":"5b94e71e-bcd1-4b51-aeaa-c047a773ca50","Type":"ContainerDied","Data":"c636faf0d78538d1b823c545125fa1151a56cfcd32d1f1f81b099c095078d229"}
Jul 28 12:30:45.860418 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:45.860378    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7655" event={"ID":"5b94e71e-bcd1-4b51-aeaa-c047a773ca50","Type":"ContainerStarted","Data":"0e637186737d5d37aebe4ea4856b0293937e81253a0bc914b2b9445297a54deb"}
Jul 28 12:30:45.887539 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:45.887479    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j7655" podStartSLOduration=8.474261376 podStartE2EDuration="37.887454418s" totalImagesPullingTime="29.413193042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:45.883408615 +0000 UTC m=+37.747658488" watchObservedRunningTime="2026-07-28 12:30:45.887454418 +0000 UTC m=+37.751704278"
Jul 28 12:30:46.721861 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:46.721664    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:46.722008 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:46.721771    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:30:46.722008 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:46.721952    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls podName:8168db1d-be3a-4555-9a2d-0876f4a02c1a nodeName:}" failed. No retries permitted until 2026-07-28 12:30:50.721936678 +0000 UTC m=+42.586186515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls") pod "dns-default-qc9cp" (UID: "8168db1d-be3a-4555-9a2d-0876f4a02c1a") : secret "dns-default-metrics-tls" not found
Jul 28 12:30:46.864305 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:46.864267    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hgnwd" event={"ID":"12e4bb44-c3f8-48b5-bc8c-a43380db0819","Type":"ContainerStarted","Data":"23a95f001427f1e69b0b9c22df106f527fff7c4547e8d9a6a8fd93e2a4c20809"}
Jul 28 12:30:46.881988 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:46.881936    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hgnwd" podStartSLOduration=2.188984283 podStartE2EDuration="4.881919807s" totalImagesPullingTime="2.692935524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:30:46.881480581 +0000 UTC m=+38.745730432" watchObservedRunningTime="2026-07-28 12:30:46.881919807 +0000 UTC m=+38.746169666"
Jul 28 12:30:50.753540 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:50.753498    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:50.754152 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:50.753739    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:30:50.754152 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:50.753825    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls podName:8168db1d-be3a-4555-9a2d-0876f4a02c1a nodeName:}" failed. No retries permitted until 2026-07-28 12:30:58.753803343 +0000 UTC m=+50.618053196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls") pod "dns-default-qc9cp" (UID: "8168db1d-be3a-4555-9a2d-0876f4a02c1a") : secret "dns-default-metrics-tls" not found
Jul 28 12:30:58.809455 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:30:58.809410    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:30:58.809967 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:58.809547    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:30:58.809967 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:30:58.809611    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls podName:8168db1d-be3a-4555-9a2d-0876f4a02c1a nodeName:}" failed. No retries permitted until 2026-07-28 12:31:14.80959674 +0000 UTC m=+66.673846576 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls") pod "dns-default-qc9cp" (UID: "8168db1d-be3a-4555-9a2d-0876f4a02c1a") : secret "dns-default-metrics-tls" not found
Jul 28 12:31:08.843316 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:08.843287    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tctq"
Jul 28 12:31:14.313572 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.313525    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:31:14.316862 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.316841    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 12:31:14.324377 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:31:14.324357    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 12:31:14.324454 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:31:14.324424    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:32:18.324405706 +0000 UTC m=+130.188655547 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : secret "metrics-daemon-secret" not found
Jul 28 12:31:14.414923 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.414889    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:31:14.418239 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.418221    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 12:31:14.429559 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.429523    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4434036-47bd-45c9-b08b-12ab9f5f84c6-original-pull-secret\") pod \"global-pull-secret-syncer-sctjj\" (UID: \"b4434036-47bd-45c9-b08b-12ab9f5f84c6\") " pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:31:14.515408 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.515364    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:31:14.518548 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.518529    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 12:31:14.528975 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.528953    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 12:31:14.538987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.538957    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx42d\" (UniqueName: \"kubernetes.io/projected/0476c580-bf8f-4896-b103-fc129ad281b9-kube-api-access-rx42d\") pod \"network-check-target-v7b58\" (UID: \"0476c580-bf8f-4896-b103-fc129ad281b9\") " pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:31:14.594944 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.594860    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sctjj"
Jul 28 12:31:14.701364 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.701338    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-klsqn\""
Jul 28 12:31:14.708998 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.708970    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:31:14.736784 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.736753    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sctjj"]
Jul 28 12:31:14.740099 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:31:14.740074    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4434036_47bd_45c9_b08b_12ab9f5f84c6.slice/crio-5c9bffdb4c1a50216e436de64b0d7367a4125a17185cbd355c67721cd87d168d WatchSource:0}: Error finding container 5c9bffdb4c1a50216e436de64b0d7367a4125a17185cbd355c67721cd87d168d: Status 404 returned error can't find the container with id 5c9bffdb4c1a50216e436de64b0d7367a4125a17185cbd355c67721cd87d168d
Jul 28 12:31:14.817244 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.817206    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:31:14.817391 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:31:14.817345    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:31:14.817431 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:31:14.817412    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls podName:8168db1d-be3a-4555-9a2d-0876f4a02c1a nodeName:}" failed. No retries permitted until 2026-07-28 12:31:46.817394951 +0000 UTC m=+98.681644792 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls") pod "dns-default-qc9cp" (UID: "8168db1d-be3a-4555-9a2d-0876f4a02c1a") : secret "dns-default-metrics-tls" not found
Jul 28 12:31:14.830309 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.830281    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v7b58"]
Jul 28 12:31:14.833692 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:31:14.833666    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0476c580_bf8f_4896_b103_fc129ad281b9.slice/crio-48d15715586634bd6379d7443827df8c6478bb8190778e1e3e3c0ad8d7c3e1e5 WatchSource:0}: Error finding container 48d15715586634bd6379d7443827df8c6478bb8190778e1e3e3c0ad8d7c3e1e5: Status 404 returned error can't find the container with id 48d15715586634bd6379d7443827df8c6478bb8190778e1e3e3c0ad8d7c3e1e5
Jul 28 12:31:14.931610 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.931524    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v7b58" event={"ID":"0476c580-bf8f-4896-b103-fc129ad281b9","Type":"ContainerStarted","Data":"48d15715586634bd6379d7443827df8c6478bb8190778e1e3e3c0ad8d7c3e1e5"}
Jul 28 12:31:14.932647 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:14.932607    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sctjj" event={"ID":"b4434036-47bd-45c9-b08b-12ab9f5f84c6","Type":"ContainerStarted","Data":"5c9bffdb4c1a50216e436de64b0d7367a4125a17185cbd355c67721cd87d168d"}
Jul 28 12:31:19.947420 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:19.947380    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v7b58" event={"ID":"0476c580-bf8f-4896-b103-fc129ad281b9","Type":"ContainerStarted","Data":"067d3467246bc8801a5d646788ff8e001670ef2719e4c0c28b708ce504e4c54d"}
Jul 28 12:31:19.948032 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:19.947510    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:31:19.948981 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:19.948954    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sctjj" event={"ID":"b4434036-47bd-45c9-b08b-12ab9f5f84c6","Type":"ContainerStarted","Data":"a461ebf6d5974544c53c661f7110843b1d79a0f48782a99866b337ded2a77a32"}
Jul 28 12:31:19.966463 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:19.966411    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v7b58" podStartSLOduration=67.662741001 podStartE2EDuration="1m11.966395213s" totalImagesPullingTime="4.303654212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:31:19.965970712 +0000 UTC m=+71.830220572" watchObservedRunningTime="2026-07-28 12:31:19.966395213 +0000 UTC m=+71.830645072"
Jul 28 12:31:19.983395 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:19.983341    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sctjj" podStartSLOduration=67.581921864 podStartE2EDuration="1m11.983324684s" totalImagesPullingTime="4.40140282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:31:19.982897115 +0000 UTC m=+71.847146975" watchObservedRunningTime="2026-07-28 12:31:19.983324684 +0000 UTC m=+71.847574540"
Jul 28 12:31:37.775120 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.775087    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"]
Jul 28 12:31:37.777881 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.777854    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.780754 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.780721    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 12:31:37.780995 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.780973    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 12:31:37.781094 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.780999    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 12:31:37.781094 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.780978    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 12:31:37.782572 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.782556    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 12:31:37.782768 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.782754    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 12:31:37.782859 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.782844    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 12:31:37.788889 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.788866    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"]
Jul 28 12:31:37.881299 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.881259    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-hub\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.881491 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.881322    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.881491 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.881358    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-ca\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.881491 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.881383    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0c3087b-df3c-4ea9-af16-0b9ef84118df-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.881491 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.881406    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzvxc\" (UniqueName: \"kubernetes.io/projected/a0c3087b-df3c-4ea9-af16-0b9ef84118df-kube-api-access-dzvxc\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.881491 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.881479    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.982686 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.982652    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-ca\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.982764 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.982725    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0c3087b-df3c-4ea9-af16-0b9ef84118df-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.982807 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.982778    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzvxc\" (UniqueName: \"kubernetes.io/projected/a0c3087b-df3c-4ea9-af16-0b9ef84118df-kube-api-access-dzvxc\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.982848 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.982808    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.982903 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.982858    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-hub\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.982950 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.982907    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.983394 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.983364    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0c3087b-df3c-4ea9-af16-0b9ef84118df-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.985172 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.985150    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-ca\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.985321 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.985300    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.985714 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.985695    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-hub\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.985784 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.985701    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a0c3087b-df3c-4ea9-af16-0b9ef84118df-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:37.994990 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:37.994957    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzvxc\" (UniqueName: \"kubernetes.io/projected/a0c3087b-df3c-4ea9-af16-0b9ef84118df-kube-api-access-dzvxc\") pod \"cluster-proxy-proxy-agent-54447d6b67-bfw6c\" (UID: \"a0c3087b-df3c-4ea9-af16-0b9ef84118df\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:38.101716 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:38.101672    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:31:38.227111 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:38.227079    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"]
Jul 28 12:31:38.230073 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:31:38.230038    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0c3087b_df3c_4ea9_af16_0b9ef84118df.slice/crio-43b738a61966f8d89ef56fa3abb13a1a507c4ecdd658b566e86b33d5094bad5f WatchSource:0}: Error finding container 43b738a61966f8d89ef56fa3abb13a1a507c4ecdd658b566e86b33d5094bad5f: Status 404 returned error can't find the container with id 43b738a61966f8d89ef56fa3abb13a1a507c4ecdd658b566e86b33d5094bad5f
Jul 28 12:31:38.998073 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:38.998036    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" event={"ID":"a0c3087b-df3c-4ea9-af16-0b9ef84118df","Type":"ContainerStarted","Data":"43b738a61966f8d89ef56fa3abb13a1a507c4ecdd658b566e86b33d5094bad5f"}
Jul 28 12:31:42.008490 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:42.008455    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" event={"ID":"a0c3087b-df3c-4ea9-af16-0b9ef84118df","Type":"ContainerStarted","Data":"51aa3d9b8c278e51c64afaa32b7599979a03a6b1e7fabac37f69440b10b880d9"}
Jul 28 12:31:43.016526 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:43.016490    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" event={"ID":"a0c3087b-df3c-4ea9-af16-0b9ef84118df","Type":"ContainerStarted","Data":"9e60d6d9fa8b4a1e9346edc47b1737401773109c61dad9522a3a63df7e53e53e"}
Jul 28 12:31:43.016526 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:43.016532    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" event={"ID":"a0c3087b-df3c-4ea9-af16-0b9ef84118df","Type":"ContainerStarted","Data":"b478b677acaa74ae2404e9922ce3cd4f7d916f75064b6961b342219f91a927a6"}
Jul 28 12:31:43.036580 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:43.036533    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" podStartSLOduration=2.3178571359999998 podStartE2EDuration="6.036518693s" totalImagesPullingTime="3.718661557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:31:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:31:43.036243965 +0000 UTC m=+94.900493826" watchObservedRunningTime="2026-07-28 12:31:43.036518693 +0000 UTC m=+94.900768530"
Jul 28 12:31:46.847694 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:46.847661    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:31:46.848067 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:31:46.847788    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:31:46.848067 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:31:46.847846    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls podName:8168db1d-be3a-4555-9a2d-0876f4a02c1a nodeName:}" failed. No retries permitted until 2026-07-28 12:32:50.847831298 +0000 UTC m=+162.712081135 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls") pod "dns-default-qc9cp" (UID: "8168db1d-be3a-4555-9a2d-0876f4a02c1a") : secret "dns-default-metrics-tls" not found
Jul 28 12:31:50.954645 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:31:50.954598    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v7b58"
Jul 28 12:32:18.358267 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:32:18.358214    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:32:18.358817 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:32:18.358375    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 12:32:18.358817 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:32:18.358443    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:34:20.358426503 +0000 UTC m=+252.222676340 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : secret "metrics-daemon-secret" not found
Jul 28 12:32:45.979745 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:32:45.979704    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qc9cp" podUID="8168db1d-be3a-4555-9a2d-0876f4a02c1a"
Jul 28 12:32:46.192673 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:32:46.192642    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:32:47.704533 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:32:47.704477    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:32:50.885501 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:32:50.885464    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:32:50.885920 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:32:50.885606    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 12:32:50.885920 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:32:50.885689    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls podName:8168db1d-be3a-4555-9a2d-0876f4a02c1a nodeName:}" failed. No retries permitted until 2026-07-28 12:34:52.8856729 +0000 UTC m=+284.749922740 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls") pod "dns-default-qc9cp" (UID: "8168db1d-be3a-4555-9a2d-0876f4a02c1a") : secret "dns-default-metrics-tls" not found
Jul 28 12:32:58.686732 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:32:58.686649    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:34:12.481258 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.481213    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b6fb7f858-lbl9l"]
Jul 28 12:34:12.483175 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.483158    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.487431 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.487408    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 12:34:12.489683 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.489663    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 12:34:12.489759 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.489667    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nfll9\""
Jul 28 12:34:12.492076 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.492057    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 12:34:12.497833 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.497816    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 12:34:12.509914 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.509890    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b6fb7f858-lbl9l"]
Jul 28 12:34:12.627461 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.627422    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-trusted-ca\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.627676 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.627480    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkcmf\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-kube-api-access-nkcmf\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.627676 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.627538    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-certificates\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.627676 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.627569    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-installation-pull-secrets\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.627676 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.627603    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-bound-sa-token\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.627676 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.627663    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-image-registry-private-configuration\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.627866 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.627686    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.627866 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.627727    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dd7015-cb0a-4294-9199-40bf7c1d1968-ca-trust-extracted\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.728769 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.728729    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-image-registry-private-configuration\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.728769 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.728769    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.728998 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:12.728940    2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:34:12.728998 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:12.728956    2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6fb7f858-lbl9l: secret "image-registry-tls" not found
Jul 28 12:34:12.729239 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:12.729025    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls podName:51dd7015-cb0a-4294-9199-40bf7c1d1968 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:13.229004269 +0000 UTC m=+245.093254124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls") pod "image-registry-6b6fb7f858-lbl9l" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968") : secret "image-registry-tls" not found
Jul 28 12:34:12.729239 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.729016    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dd7015-cb0a-4294-9199-40bf7c1d1968-ca-trust-extracted\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.729239 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.729075    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-trusted-ca\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.729239 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.729123    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkcmf\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-kube-api-access-nkcmf\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.729239 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.729160    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-certificates\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.729239 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.729187    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-installation-pull-secrets\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.729239 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.729226    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-bound-sa-token\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.729541 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.729322    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dd7015-cb0a-4294-9199-40bf7c1d1968-ca-trust-extracted\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.729951 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.729924    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-certificates\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.730036 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.730019    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-trusted-ca\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.731388 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.731345    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-image-registry-private-configuration\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.731551 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.731531    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-installation-pull-secrets\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.738011 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.737987    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-bound-sa-token\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:12.738180 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:12.738153    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkcmf\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-kube-api-access-nkcmf\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:13.232552 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:13.232506    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:13.232739 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:13.232684    2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:34:13.232739 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:13.232705    2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6fb7f858-lbl9l: secret "image-registry-tls" not found
Jul 28 12:34:13.232815 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:13.232773    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls podName:51dd7015-cb0a-4294-9199-40bf7c1d1968 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:14.232757221 +0000 UTC m=+246.097007058 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls") pod "image-registry-6b6fb7f858-lbl9l" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968") : secret "image-registry-tls" not found
Jul 28 12:34:14.239876 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:14.239820    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:14.240265 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:14.239978    2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:34:14.240265 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:14.239992    2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6fb7f858-lbl9l: secret "image-registry-tls" not found
Jul 28 12:34:14.240265 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:14.240058    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls podName:51dd7015-cb0a-4294-9199-40bf7c1d1968 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:16.240041057 +0000 UTC m=+248.104290900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls") pod "image-registry-6b6fb7f858-lbl9l" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968") : secret "image-registry-tls" not found
Jul 28 12:34:16.252989 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.252949    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:16.253338 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:16.253097    2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:34:16.253338 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:16.253116    2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6fb7f858-lbl9l: secret "image-registry-tls" not found
Jul 28 12:34:16.253338 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:16.253172    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls podName:51dd7015-cb0a-4294-9199-40bf7c1d1968 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:20.253156095 +0000 UTC m=+252.117405932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls") pod "image-registry-6b6fb7f858-lbl9l" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968") : secret "image-registry-tls" not found
Jul 28 12:34:16.644336 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.644301    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j9sxv"]
Jul 28 12:34:16.647582 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.647565    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.650494 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.650471    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 12:34:16.650768 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.650749    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cv5n6\""
Jul 28 12:34:16.650856 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.650749    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 12:34:16.652080 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.652057    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 12:34:16.652174 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.652128    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 12:34:16.659425 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.659404    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j9sxv"]
Jul 28 12:34:16.758101 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.758058    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.758101 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.758103    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.758380 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.758199    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d4f01c5-13ee-42a1-9969-28b68e374307-crio-socket\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.758380 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.758235    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8lwm\" (UniqueName: \"kubernetes.io/projected/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-api-access-x8lwm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.758380 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.758255    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d4f01c5-13ee-42a1-9969-28b68e374307-data-volume\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.859561 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.859531    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.859745 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.859603    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d4f01c5-13ee-42a1-9969-28b68e374307-crio-socket\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.859745 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.859669    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8lwm\" (UniqueName: \"kubernetes.io/projected/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-api-access-x8lwm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.859745 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:16.859680    2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:34:16.859745 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.859701    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d4f01c5-13ee-42a1-9969-28b68e374307-data-volume\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.859745 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.859704    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d4f01c5-13ee-42a1-9969-28b68e374307-crio-socket\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.859928 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:16.859756    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:17.35973784 +0000 UTC m=+249.223987685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : configmap "kube-rbac-proxy" not found
Jul 28 12:34:16.859928 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.859805    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.859928 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:16.859915    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:16.860035 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:16.859960    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:17.359946208 +0000 UTC m=+249.224196067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:16.860035 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.860024    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d4f01c5-13ee-42a1-9969-28b68e374307-data-volume\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:16.869266 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:16.869236    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8lwm\" (UniqueName: \"kubernetes.io/projected/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-api-access-x8lwm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:17.364045 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:17.363991    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:17.364045 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:17.364050    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:17.364519 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:17.364149    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:17.364519 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:17.364215    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:18.364199515 +0000 UTC m=+250.228449352 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:17.364519 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:17.364150    2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:34:17.364519 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:17.364310    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:18.364293721 +0000 UTC m=+250.228543564 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : configmap "kube-rbac-proxy" not found
Jul 28 12:34:18.372193 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:18.372140    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:18.372193 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:18.372198    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:18.372735 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:18.372304    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:18.372735 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:18.372330    2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:34:18.372735 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:18.372365    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:20.372350555 +0000 UTC m=+252.236600396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:18.372735 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:18.372378    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:20.372372454 +0000 UTC m=+252.236622295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : configmap "kube-rbac-proxy" not found
Jul 28 12:34:19.227585 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:19.227553    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bhst2_d917c83e-d627-4099-ba45-58674437b5a8/dns-node-resolver/0.log"
Jul 28 12:34:20.284076 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:20.284039    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:20.284447 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:20.284194    2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 12:34:20.284447 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:20.284213    2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6fb7f858-lbl9l: secret "image-registry-tls" not found
Jul 28 12:34:20.284447 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:20.284284    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls podName:51dd7015-cb0a-4294-9199-40bf7c1d1968 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:28.284265438 +0000 UTC m=+260.148515280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls") pod "image-registry-6b6fb7f858-lbl9l" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968") : secret "image-registry-tls" not found
Jul 28 12:34:20.384585 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:20.384545    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:20.384725 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:20.384591    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:20.384725 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:20.384648    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:34:20.384795 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:20.384733    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:20.384826 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:20.384803    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:24.384787088 +0000 UTC m=+256.249036925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:20.384826 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:20.384733    2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:34:20.384895 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:20.384737    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 12:34:20.384895 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:20.384835    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:24.384829214 +0000 UTC m=+256.249079051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : configmap "kube-rbac-proxy" not found
Jul 28 12:34:20.384895 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:20.384891    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs podName:5a25858a-ae63-4104-b1a4-33ac27e882df nodeName:}" failed. No retries permitted until 2026-07-28 12:36:22.384879516 +0000 UTC m=+374.249129354 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs") pod "network-metrics-daemon-mg9wl" (UID: "5a25858a-ae63-4104-b1a4-33ac27e882df") : secret "metrics-daemon-secret" not found
Jul 28 12:34:20.428520 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:20.428494    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qbgl6_f1b592e4-fbbf-4fe8-aa78-c10e97343d81/node-ca/0.log"
Jul 28 12:34:21.031941 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:21.031907    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hgnwd_12e4bb44-c3f8-48b5-bc8c-a43380db0819/serve-healthcheck-canary/0.log"
Jul 28 12:34:24.414138 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:24.414085    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:24.414138 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:24.414140    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:24.414581 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:24.414248    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:24.414581 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:24.414264    2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 12:34:24.414581 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:24.414312    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:32.414297275 +0000 UTC m=+264.278547112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:24.414581 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:24.414328    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:32.414321376 +0000 UTC m=+264.278571212 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : configmap "kube-rbac-proxy" not found
Jul 28 12:34:28.347328 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:28.347229    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:28.349796 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:28.349775    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") pod \"image-registry-6b6fb7f858-lbl9l\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") " pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:28.393029 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:28.392996    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:28.521048 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:28.520890    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b6fb7f858-lbl9l"]
Jul 28 12:34:28.523485 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:34:28.523460    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51dd7015_cb0a_4294_9199_40bf7c1d1968.slice/crio-71dd4b24e975b02b7f0aeec18d3d217d9b8deacfd57e105b09f61f511b4a9996 WatchSource:0}: Error finding container 71dd4b24e975b02b7f0aeec18d3d217d9b8deacfd57e105b09f61f511b4a9996: Status 404 returned error can't find the container with id 71dd4b24e975b02b7f0aeec18d3d217d9b8deacfd57e105b09f61f511b4a9996
Jul 28 12:34:29.472247 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:29.472218    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l" event={"ID":"51dd7015-cb0a-4294-9199-40bf7c1d1968","Type":"ContainerStarted","Data":"c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833"}
Jul 28 12:34:29.472703 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:29.472268    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l" event={"ID":"51dd7015-cb0a-4294-9199-40bf7c1d1968","Type":"ContainerStarted","Data":"71dd4b24e975b02b7f0aeec18d3d217d9b8deacfd57e105b09f61f511b4a9996"}
Jul 28 12:34:29.472703 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:29.472306    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:29.491395 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:29.491351    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l" podStartSLOduration=17.491321047 podStartE2EDuration="17.491321047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:34:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:34:29.490424751 +0000 UTC m=+261.354674611" watchObservedRunningTime="2026-07-28 12:34:29.491321047 +0000 UTC m=+261.355570907"
Jul 28 12:34:32.473071 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:32.473033    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:32.473071 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:32.473080    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:32.473538 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:32.473219    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:32.473538 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:32.473304    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls podName:9d4f01c5-13ee-42a1-9969-28b68e374307 nodeName:}" failed. No retries permitted until 2026-07-28 12:34:48.473285675 +0000 UTC m=+280.337535513 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j9sxv" (UID: "9d4f01c5-13ee-42a1-9969-28b68e374307") : secret "insights-runtime-extractor-tls" not found
Jul 28 12:34:32.473538 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:32.473531    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d4f01c5-13ee-42a1-9969-28b68e374307-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:42.708641 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.708582    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b6fb7f858-lbl9l"]
Jul 28 12:34:42.799685 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.799649    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bc8d7748d-6d7pj"]
Jul 28 12:34:42.804497 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.804479    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:42.813268 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.813245    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bc8d7748d-6d7pj"]
Jul 28 12:34:42.947095 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.947058    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2924e840-8e0a-4f46-b95a-5c5673c022c8-image-registry-private-configuration\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:42.947095 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.947099    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2924e840-8e0a-4f46-b95a-5c5673c022c8-trusted-ca\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:42.947344 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.947126    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2924e840-8e0a-4f46-b95a-5c5673c022c8-installation-pull-secrets\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:42.947344 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.947154    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7php\" (UniqueName: \"kubernetes.io/projected/2924e840-8e0a-4f46-b95a-5c5673c022c8-kube-api-access-k7php\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:42.947344 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.947179    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2924e840-8e0a-4f46-b95a-5c5673c022c8-ca-trust-extracted\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:42.947344 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.947199    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2924e840-8e0a-4f46-b95a-5c5673c022c8-registry-certificates\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:42.947344 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.947279    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2924e840-8e0a-4f46-b95a-5c5673c022c8-bound-sa-token\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:42.947344 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:42.947335    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2924e840-8e0a-4f46-b95a-5c5673c022c8-registry-tls\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.048616 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.048531    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2924e840-8e0a-4f46-b95a-5c5673c022c8-trusted-ca\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.048616 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.048571    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2924e840-8e0a-4f46-b95a-5c5673c022c8-installation-pull-secrets\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.048616 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.048590    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7php\" (UniqueName: \"kubernetes.io/projected/2924e840-8e0a-4f46-b95a-5c5673c022c8-kube-api-access-k7php\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.048616 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.048609    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2924e840-8e0a-4f46-b95a-5c5673c022c8-ca-trust-extracted\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.048922 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.048820    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2924e840-8e0a-4f46-b95a-5c5673c022c8-registry-certificates\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.048922 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.048898    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2924e840-8e0a-4f46-b95a-5c5673c022c8-bound-sa-token\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.049024 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.048947    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2924e840-8e0a-4f46-b95a-5c5673c022c8-registry-tls\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.049024 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.048996    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2924e840-8e0a-4f46-b95a-5c5673c022c8-ca-trust-extracted\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.049128 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.049047    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2924e840-8e0a-4f46-b95a-5c5673c022c8-image-registry-private-configuration\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.049573 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.049548    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2924e840-8e0a-4f46-b95a-5c5673c022c8-registry-certificates\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.049866 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.049827    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2924e840-8e0a-4f46-b95a-5c5673c022c8-trusted-ca\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.051332 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.051305    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2924e840-8e0a-4f46-b95a-5c5673c022c8-image-registry-private-configuration\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.051438 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.051339    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2924e840-8e0a-4f46-b95a-5c5673c022c8-installation-pull-secrets\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.051502 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.051490    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2924e840-8e0a-4f46-b95a-5c5673c022c8-registry-tls\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.057666 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.057644    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7php\" (UniqueName: \"kubernetes.io/projected/2924e840-8e0a-4f46-b95a-5c5673c022c8-kube-api-access-k7php\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.057998 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.057977    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2924e840-8e0a-4f46-b95a-5c5673c022c8-bound-sa-token\") pod \"image-registry-bc8d7748d-6d7pj\" (UID: \"2924e840-8e0a-4f46-b95a-5c5673c022c8\") " pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.112768 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.112728    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.239647 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.239606    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bc8d7748d-6d7pj"]
Jul 28 12:34:43.241954 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:34:43.241922    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2924e840_8e0a_4f46_b95a_5c5673c022c8.slice/crio-e92f50003b27ec9d5ae3c874efb9e5b0600b508373061dd4e538ddf84792bdeb WatchSource:0}: Error finding container e92f50003b27ec9d5ae3c874efb9e5b0600b508373061dd4e538ddf84792bdeb: Status 404 returned error can't find the container with id e92f50003b27ec9d5ae3c874efb9e5b0600b508373061dd4e538ddf84792bdeb
Jul 28 12:34:43.513227 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.513187    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj" event={"ID":"2924e840-8e0a-4f46-b95a-5c5673c022c8","Type":"ContainerStarted","Data":"6124f5475b632096615fd1025d11fa91f5fbf344a406216b6d1c2c6c6670cae8"}
Jul 28 12:34:43.513385 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.513286    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj" event={"ID":"2924e840-8e0a-4f46-b95a-5c5673c022c8","Type":"ContainerStarted","Data":"e92f50003b27ec9d5ae3c874efb9e5b0600b508373061dd4e538ddf84792bdeb"}
Jul 28 12:34:43.513385 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.513314    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:34:43.533326 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:43.533276    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj" podStartSLOduration=1.533260456 podStartE2EDuration="1.533260456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:34:43.532141782 +0000 UTC m=+275.396391653" watchObservedRunningTime="2026-07-28 12:34:43.533260456 +0000 UTC m=+275.397510314"
Jul 28 12:34:48.485785 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:48.485745    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:48.488174 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:48.488151    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d4f01c5-13ee-42a1-9969-28b68e374307-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j9sxv\" (UID: \"9d4f01c5-13ee-42a1-9969-28b68e374307\") " pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:48.756467 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:48.756362    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j9sxv"
Jul 28 12:34:48.880559 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:48.880530    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j9sxv"]
Jul 28 12:34:48.883664 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:34:48.883617    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d4f01c5_13ee_42a1_9969_28b68e374307.slice/crio-2344f29d897c71c4571b90675b99d25c083ef1cf188d9b0ca4bbc9466cd7695a WatchSource:0}: Error finding container 2344f29d897c71c4571b90675b99d25c083ef1cf188d9b0ca4bbc9466cd7695a: Status 404 returned error can't find the container with id 2344f29d897c71c4571b90675b99d25c083ef1cf188d9b0ca4bbc9466cd7695a
Jul 28 12:34:49.193075 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:34:49.193032    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qc9cp" podUID="8168db1d-be3a-4555-9a2d-0876f4a02c1a"
Jul 28 12:34:49.533501 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:49.533416    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:34:49.533985 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:49.533798    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j9sxv" event={"ID":"9d4f01c5-13ee-42a1-9969-28b68e374307","Type":"ContainerStarted","Data":"2344f29d897c71c4571b90675b99d25c083ef1cf188d9b0ca4bbc9466cd7695a"}
Jul 28 12:34:52.714773 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:52.714743    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:34:52.921267 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:52.921229    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:34:52.924009 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:52.923982    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8168db1d-be3a-4555-9a2d-0876f4a02c1a-metrics-tls\") pod \"dns-default-qc9cp\" (UID: \"8168db1d-be3a-4555-9a2d-0876f4a02c1a\") " pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:34:53.137088 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:53.137055    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bstgx\""
Jul 28 12:34:53.144820 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:53.144790    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:34:53.924881 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:53.924858    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qc9cp"]
Jul 28 12:34:53.928708 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:34:53.928687    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8168db1d_be3a_4555_9a2d_0876f4a02c1a.slice/crio-b7cd647d79bc59aca96bc52dd83a8eea853a5886003c8a0312723fa2cd9325dd WatchSource:0}: Error finding container b7cd647d79bc59aca96bc52dd83a8eea853a5886003c8a0312723fa2cd9325dd: Status 404 returned error can't find the container with id b7cd647d79bc59aca96bc52dd83a8eea853a5886003c8a0312723fa2cd9325dd
Jul 28 12:34:54.553265 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:54.553217    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j9sxv" event={"ID":"9d4f01c5-13ee-42a1-9969-28b68e374307","Type":"ContainerStarted","Data":"33383d0b6d85460c28df835bcdc2442ed0e833cdd71cad9a9b3c2e320ccf50be"}
Jul 28 12:34:54.554982 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:54.554908    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qc9cp" event={"ID":"8168db1d-be3a-4555-9a2d-0876f4a02c1a","Type":"ContainerStarted","Data":"b7cd647d79bc59aca96bc52dd83a8eea853a5886003c8a0312723fa2cd9325dd"}
Jul 28 12:34:58.569431 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:58.569355    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qc9cp" event={"ID":"8168db1d-be3a-4555-9a2d-0876f4a02c1a","Type":"ContainerStarted","Data":"5aa70641942a6f678d5dfd04d44a80dbb2586da9db08a7a03cc7c20e3c1f1964"}
Jul 28 12:34:58.569431 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:58.569420    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qc9cp" event={"ID":"8168db1d-be3a-4555-9a2d-0876f4a02c1a","Type":"ContainerStarted","Data":"d408d573532a5e13104311db0eed846a43d252655aa0ae0d29a5479dd04f25ec"}
Jul 28 12:34:58.569987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:58.569468    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:34:58.571293 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:58.571260    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j9sxv" event={"ID":"9d4f01c5-13ee-42a1-9969-28b68e374307","Type":"ContainerStarted","Data":"535b41c673f6cc46c2ca93befcaccfe9c326f7dd0c275071a7cbaca815b49948"}
Jul 28 12:34:58.592153 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:34:58.592086    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qc9cp" podStartSLOduration=252.810266321 podStartE2EDuration="4m16.592068722s" totalImagesPullingTime="3.781802401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:34:58.591076896 +0000 UTC m=+290.455326757" watchObservedRunningTime="2026-07-28 12:34:58.592068722 +0000 UTC m=+290.456318581"
Jul 28 12:35:01.584388 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:01.584354    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j9sxv" event={"ID":"9d4f01c5-13ee-42a1-9969-28b68e374307","Type":"ContainerStarted","Data":"d9168552c8a14c518f863e65ec138ff92305139f5bb61ff9e3077dbe909ee4b8"}
Jul 28 12:35:01.604480 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:01.604440    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j9sxv" podStartSLOduration=33.367985102 podStartE2EDuration="45.604424645s" totalImagesPullingTime="12.236439543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:34:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:35:01.60249816 +0000 UTC m=+293.466748043" watchObservedRunningTime="2026-07-28 12:35:01.604424645 +0000 UTC m=+293.468674505"
Jul 28 12:35:01.687943 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:35:01.687897    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mg9wl" podUID="5a25858a-ae63-4104-b1a4-33ac27e882df"
Jul 28 12:35:04.521300 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:04.521267    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bc8d7748d-6d7pj"
Jul 28 12:35:07.727894 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:07.727836    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l" podUID="51dd7015-cb0a-4294-9199-40bf7c1d1968" containerName="registry" containerID="cri-o://c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833" gracePeriod=30
Jul 28 12:35:07.958242 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:07.958219    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:35:08.028599 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.028521    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkcmf\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-kube-api-access-nkcmf\") pod \"51dd7015-cb0a-4294-9199-40bf7c1d1968\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") "
Jul 28 12:35:08.028599 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.028564    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-certificates\") pod \"51dd7015-cb0a-4294-9199-40bf7c1d1968\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") "
Jul 28 12:35:08.028599 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.028593    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dd7015-cb0a-4294-9199-40bf7c1d1968-ca-trust-extracted\") pod \"51dd7015-cb0a-4294-9199-40bf7c1d1968\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") "
Jul 28 12:35:08.028884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.028637    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") pod \"51dd7015-cb0a-4294-9199-40bf7c1d1968\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") "
Jul 28 12:35:08.028884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.028758    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-bound-sa-token\") pod \"51dd7015-cb0a-4294-9199-40bf7c1d1968\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") "
Jul 28 12:35:08.028884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.028810    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-trusted-ca\") pod \"51dd7015-cb0a-4294-9199-40bf7c1d1968\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") "
Jul 28 12:35:08.028884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.028843    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-installation-pull-secrets\") pod \"51dd7015-cb0a-4294-9199-40bf7c1d1968\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") "
Jul 28 12:35:08.028884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.028873    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-image-registry-private-configuration\") pod \"51dd7015-cb0a-4294-9199-40bf7c1d1968\" (UID: \"51dd7015-cb0a-4294-9199-40bf7c1d1968\") "
Jul 28 12:35:08.029141 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.029117    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "51dd7015-cb0a-4294-9199-40bf7c1d1968" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:35:08.029238 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.029204    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "51dd7015-cb0a-4294-9199-40bf7c1d1968" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:35:08.031447 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.031413    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "51dd7015-cb0a-4294-9199-40bf7c1d1968" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:35:08.031560 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.031439    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "51dd7015-cb0a-4294-9199-40bf7c1d1968" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:35:08.031560 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.031470    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-kube-api-access-nkcmf" (OuterVolumeSpecName: "kube-api-access-nkcmf") pod "51dd7015-cb0a-4294-9199-40bf7c1d1968" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968"). InnerVolumeSpecName "kube-api-access-nkcmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:35:08.031702 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.031592    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "51dd7015-cb0a-4294-9199-40bf7c1d1968" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:35:08.031761 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.031700    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "51dd7015-cb0a-4294-9199-40bf7c1d1968" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:35:08.037291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.037268    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dd7015-cb0a-4294-9199-40bf7c1d1968-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "51dd7015-cb0a-4294-9199-40bf7c1d1968" (UID: "51dd7015-cb0a-4294-9199-40bf7c1d1968"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:35:08.129566 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.129525    2549 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-certificates\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:35:08.129566 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.129555    2549 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dd7015-cb0a-4294-9199-40bf7c1d1968-ca-trust-extracted\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:35:08.129566 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.129565    2549 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-registry-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:35:08.129566 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.129573    2549 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-bound-sa-token\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:35:08.129566 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.129581    2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dd7015-cb0a-4294-9199-40bf7c1d1968-trusted-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:35:08.129873 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.129589    2549 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-installation-pull-secrets\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:35:08.129873 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.129598    2549 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dd7015-cb0a-4294-9199-40bf7c1d1968-image-registry-private-configuration\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:35:08.129873 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.129607    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkcmf\" (UniqueName: \"kubernetes.io/projected/51dd7015-cb0a-4294-9199-40bf7c1d1968-kube-api-access-nkcmf\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:35:08.395477 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.395443    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"]
Jul 28 12:35:08.395675 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.395663    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51dd7015-cb0a-4294-9199-40bf7c1d1968" containerName="registry"
Jul 28 12:35:08.395722 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.395676    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="51dd7015-cb0a-4294-9199-40bf7c1d1968" containerName="registry"
Jul 28 12:35:08.395755 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.395721    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="51dd7015-cb0a-4294-9199-40bf7c1d1968" containerName="registry"
Jul 28 12:35:08.399109 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.399093    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.402104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.402081    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ds785\""
Jul 28 12:35:08.402534 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.402518    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 12:35:08.403700 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.403682    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 12:35:08.403700 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.403693    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 12:35:08.403834 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.403696    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 12:35:08.403834 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.403696    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 12:35:08.422174 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.422144    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"]
Jul 28 12:35:08.432340 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.432317    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c38306ff-de52-4ef3-932f-ac3b727114c3-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.432461 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.432371    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c38306ff-de52-4ef3-932f-ac3b727114c3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.432461 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.432396    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr2fq\" (UniqueName: \"kubernetes.io/projected/c38306ff-de52-4ef3-932f-ac3b727114c3-kube-api-access-vr2fq\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.432461 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.432442    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c38306ff-de52-4ef3-932f-ac3b727114c3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.436078 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.436056    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"]
Jul 28 12:35:08.441449 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.441353    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.445075 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.445057    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 12:35:08.445288 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.445272    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 12:35:08.445379 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.445361    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8stcf\""
Jul 28 12:35:08.445489 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.445474    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 12:35:08.456725 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.456704    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"]
Jul 28 12:35:08.465615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.465591    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c2p9n"]
Jul 28 12:35:08.468017 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.467997    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.470865 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.470846    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 12:35:08.471082 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.471065    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xd6th\""
Jul 28 12:35:08.471143 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.471072    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 12:35:08.471227 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.471213    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 12:35:08.524512 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.524481    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:35:08.525064 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.525048    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:35:08.529466 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.529444    2549 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 12:35:08.532805 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.532783    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lglg4\" (UniqueName: \"kubernetes.io/projected/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-api-access-lglg4\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.532924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.532812    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76761059-1f76-43f6-8e6f-36ef294eba40-sys\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.532924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.532857    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr2fq\" (UniqueName: \"kubernetes.io/projected/c38306ff-de52-4ef3-932f-ac3b727114c3-kube-api-access-vr2fq\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.532924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.532887    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-textfile\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.533137 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.532927    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c38306ff-de52-4ef3-932f-ac3b727114c3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.533137 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.532972    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.533137 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533005    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/76761059-1f76-43f6-8e6f-36ef294eba40-root\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.533137 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533035    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-tls\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.533137 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533054    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c38306ff-de52-4ef3-932f-ac3b727114c3-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.533137 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533082    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-accelerators-collector-config\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.533137 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533108    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-wtmp\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.533523 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533143    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.533523 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533167    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.533523 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533237    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.533523 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533295    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9pmq\" (UniqueName: \"kubernetes.io/projected/76761059-1f76-43f6-8e6f-36ef294eba40-kube-api-access-h9pmq\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.533523 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533326    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.533523 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533387    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c38306ff-de52-4ef3-932f-ac3b727114c3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.533523 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533423    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.533523 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.533450    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76761059-1f76-43f6-8e6f-36ef294eba40-metrics-client-ca\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.535944 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.535926    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 12:35:08.535944 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.535939    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 12:35:08.536088 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.535930    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 12:35:08.542563 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.542550    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 12:35:08.543582 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:35:08.543570    2549 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 28 12:35:08.543649 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:35:08.543615    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c38306ff-de52-4ef3-932f-ac3b727114c3-openshift-state-metrics-tls podName:c38306ff-de52-4ef3-932f-ac3b727114c3 nodeName:}" failed. No retries permitted until 2026-07-28 12:35:09.0436006 +0000 UTC m=+300.907850437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c38306ff-de52-4ef3-932f-ac3b727114c3-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-hrfss" (UID: "c38306ff-de52-4ef3-932f-ac3b727114c3") : secret "openshift-state-metrics-tls" not found
Jul 28 12:35:08.544149 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.544129    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c38306ff-de52-4ef3-932f-ac3b727114c3-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.545795 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.545780    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c38306ff-de52-4ef3-932f-ac3b727114c3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.553501 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.553487    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 12:35:08.563182 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.563150    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr2fq\" (UniqueName: \"kubernetes.io/projected/c38306ff-de52-4ef3-932f-ac3b727114c3-kube-api-access-vr2fq\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:08.576807 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.576788    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qc9cp"
Jul 28 12:35:08.606597 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.606562    2549 generic.go:356] "Generic (PLEG): container finished" podID="51dd7015-cb0a-4294-9199-40bf7c1d1968" containerID="c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833" exitCode=0
Jul 28 12:35:08.606743 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.606662    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l"
Jul 28 12:35:08.606743 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.606693    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l" event={"ID":"51dd7015-cb0a-4294-9199-40bf7c1d1968","Type":"ContainerDied","Data":"c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833"}
Jul 28 12:35:08.606846 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.606751    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6fb7f858-lbl9l" event={"ID":"51dd7015-cb0a-4294-9199-40bf7c1d1968","Type":"ContainerDied","Data":"71dd4b24e975b02b7f0aeec18d3d217d9b8deacfd57e105b09f61f511b4a9996"}
Jul 28 12:35:08.606846 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.606804    2549 scope.go:117] "RemoveContainer" containerID="c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833"
Jul 28 12:35:08.616069 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.616044    2549 scope.go:117] "RemoveContainer" containerID="c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833"
Jul 28 12:35:08.616382 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:35:08.616352    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833\": container with ID starting with c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833 not found: ID does not exist" containerID="c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833"
Jul 28 12:35:08.616490 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.616394    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833"} err="failed to get container status \"c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833\": rpc error: code = NotFound desc = could not find container \"c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833\": container with ID starting with c844af3312672ed62cd4adb4d1e412d6215b347cf2d1cf8715ef43ae107ba833 not found: ID does not exist"
Jul 28 12:35:08.633831 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.633800    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-wtmp\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.633997 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.633858    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.633997 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.633885    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.633997 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.633914    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.633997 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.633974    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9pmq\" (UniqueName: \"kubernetes.io/projected/76761059-1f76-43f6-8e6f-36ef294eba40-kube-api-access-h9pmq\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634228 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634011    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.634228 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634027    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-wtmp\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634228 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634046    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.634228 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634072    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76761059-1f76-43f6-8e6f-36ef294eba40-metrics-client-ca\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634228 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634100    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lglg4\" (UniqueName: \"kubernetes.io/projected/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-api-access-lglg4\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.634228 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634126    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76761059-1f76-43f6-8e6f-36ef294eba40-sys\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634228 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634158    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-textfile\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634228 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634224    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.634597 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634258    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/76761059-1f76-43f6-8e6f-36ef294eba40-root\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634597 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634280    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-tls\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634597 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634299    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-accelerators-collector-config\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634597 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634395    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76761059-1f76-43f6-8e6f-36ef294eba40-sys\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634819 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634687    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-textfile\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.634819 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634704    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.635281 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.634964    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/76761059-1f76-43f6-8e6f-36ef294eba40-root\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.635558 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.635536    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76761059-1f76-43f6-8e6f-36ef294eba40-metrics-client-ca\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.635661 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.635549    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.635661 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.635599    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b6fb7f858-lbl9l"]
Jul 28 12:35:08.637660 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.637613    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 12:35:08.637741 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.637674    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 12:35:08.637953 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.637938    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 12:35:08.638165 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.638141    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 12:35:08.638261 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.638175    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 12:35:08.638328 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.638258    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 12:35:08.641407 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.641387    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b6fb7f858-lbl9l"]
Jul 28 12:35:08.645103 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.645076    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-accelerators-collector-config\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.645200 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:35:08.645186    2549 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 28 12:35:08.645282 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:35:08.645267    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-tls podName:7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f nodeName:}" failed. No retries permitted until 2026-07-28 12:35:09.145246863 +0000 UTC m=+301.009496703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-t2dw4" (UID: "7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f") : secret "kube-state-metrics-tls" not found
Jul 28 12:35:08.645574 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.645511    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.647591 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.647570    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.647803 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.647783    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.647959 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.647945    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lglg4\" (UniqueName: \"kubernetes.io/projected/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-api-access-lglg4\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:08.647999 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.647954    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76761059-1f76-43f6-8e6f-36ef294eba40-node-exporter-tls\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.648936 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.648915    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9pmq\" (UniqueName: \"kubernetes.io/projected/76761059-1f76-43f6-8e6f-36ef294eba40-kube-api-access-h9pmq\") pod \"node-exporter-c2p9n\" (UID: \"76761059-1f76-43f6-8e6f-36ef294eba40\") " pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.693553 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.693423    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51dd7015-cb0a-4294-9199-40bf7c1d1968" path="/var/lib/kubelet/pods/51dd7015-cb0a-4294-9199-40bf7c1d1968/volumes"
Jul 28 12:35:08.780155 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.780132    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xd6th\""
Jul 28 12:35:08.787923 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:08.787902    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c2p9n"
Jul 28 12:35:08.797744 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:35:08.797719    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76761059_1f76_43f6_8e6f_36ef294eba40.slice/crio-b89d96a9f0311d8c5cd4e914e12403779e50803478ba71a3a58487d97122e38c WatchSource:0}: Error finding container b89d96a9f0311d8c5cd4e914e12403779e50803478ba71a3a58487d97122e38c: Status 404 returned error can't find the container with id b89d96a9f0311d8c5cd4e914e12403779e50803478ba71a3a58487d97122e38c
Jul 28 12:35:09.139007 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.138973    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c38306ff-de52-4ef3-932f-ac3b727114c3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:09.141389 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.141361    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c38306ff-de52-4ef3-932f-ac3b727114c3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-hrfss\" (UID: \"c38306ff-de52-4ef3-932f-ac3b727114c3\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:09.240062 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.240020    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:09.242482 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.242453    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-t2dw4\" (UID: \"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:09.310835 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.310802    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ds785\""
Jul 28 12:35:09.319084 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.319051    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"
Jul 28 12:35:09.352976 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.352949    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8stcf\""
Jul 28 12:35:09.360694 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.360665    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"
Jul 28 12:35:09.460646 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.460580    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss"]
Jul 28 12:35:09.467916 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:35:09.467195    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc38306ff_de52_4ef3_932f_ac3b727114c3.slice/crio-28e2189eaf46861dae8eb8e5ad9d1b43acf76ef5e0b624661b144c75b55830f1 WatchSource:0}: Error finding container 28e2189eaf46861dae8eb8e5ad9d1b43acf76ef5e0b624661b144c75b55830f1: Status 404 returned error can't find the container with id 28e2189eaf46861dae8eb8e5ad9d1b43acf76ef5e0b624661b144c75b55830f1
Jul 28 12:35:09.481836 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.481804    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 12:35:09.484937 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.484916    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.488136 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488101    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 12:35:09.488242 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488147    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 12:35:09.488242 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488100    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 12:35:09.488242 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488169    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 12:35:09.488408 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488107    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hfskc\""
Jul 28 12:35:09.488408 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488362    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 12:35:09.488749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488598    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 12:35:09.488749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488611    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 12:35:09.488749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488659    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 12:35:09.488749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.488704    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 12:35:09.501413 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.501391    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 12:35:09.523893 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.523866    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4"]
Jul 28 12:35:09.526963 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:35:09.526937    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c18d8b7_57cf_4d3a_a20a_7b3d02b3e60f.slice/crio-80bf40c6cfcec93e4be3d5bc5410d8ae1bdb7dc1298d478cc2723dced4736db0 WatchSource:0}: Error finding container 80bf40c6cfcec93e4be3d5bc5410d8ae1bdb7dc1298d478cc2723dced4736db0: Status 404 returned error can't find the container with id 80bf40c6cfcec93e4be3d5bc5410d8ae1bdb7dc1298d478cc2723dced4736db0
Jul 28 12:35:09.543356 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543335    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.543516 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543367    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.543516 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543397    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f690b07a-3ac7-44a1-b208-f004c7220c9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.543516 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543419    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-web-config\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.543516 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543500    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.543778 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543575    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f690b07a-3ac7-44a1-b208-f004c7220c9e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.543778 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543603    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmj5d\" (UniqueName: \"kubernetes.io/projected/f690b07a-3ac7-44a1-b208-f004c7220c9e-kube-api-access-wmj5d\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.543778 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543647    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.543778 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543721    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.543778 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543756    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.544012 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543787    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f690b07a-3ac7-44a1-b208-f004c7220c9e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.544012 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543818    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f690b07a-3ac7-44a1-b208-f004c7220c9e-config-out\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.544012 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.543871    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f690b07a-3ac7-44a1-b208-f004c7220c9e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.610977 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.610946    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss" event={"ID":"c38306ff-de52-4ef3-932f-ac3b727114c3","Type":"ContainerStarted","Data":"3ec35fb9e3cea64ce94ef48b0a57cdd16ad6e683bb91a3d8d34dac69cc3400f2"}
Jul 28 12:35:09.611080 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.611016    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss" event={"ID":"c38306ff-de52-4ef3-932f-ac3b727114c3","Type":"ContainerStarted","Data":"8ba83754adf9314cba29a6b4d84b7840a8348ca24e88778163ad42b822c31b60"}
Jul 28 12:35:09.611080 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.611048    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss" event={"ID":"c38306ff-de52-4ef3-932f-ac3b727114c3","Type":"ContainerStarted","Data":"28e2189eaf46861dae8eb8e5ad9d1b43acf76ef5e0b624661b144c75b55830f1"}
Jul 28 12:35:09.612965 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.612944    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2p9n" event={"ID":"76761059-1f76-43f6-8e6f-36ef294eba40","Type":"ContainerStarted","Data":"b89d96a9f0311d8c5cd4e914e12403779e50803478ba71a3a58487d97122e38c"}
Jul 28 12:35:09.614789 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.614768    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4" event={"ID":"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f","Type":"ContainerStarted","Data":"80bf40c6cfcec93e4be3d5bc5410d8ae1bdb7dc1298d478cc2723dced4736db0"}
Jul 28 12:35:09.644882 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.644852    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f690b07a-3ac7-44a1-b208-f004c7220c9e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.644889    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmj5d\" (UniqueName: \"kubernetes.io/projected/f690b07a-3ac7-44a1-b208-f004c7220c9e-kube-api-access-wmj5d\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.644914    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.644946    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.644975    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645003    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f690b07a-3ac7-44a1-b208-f004c7220c9e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645275 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645030    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f690b07a-3ac7-44a1-b208-f004c7220c9e-config-out\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645275 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645249    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f690b07a-3ac7-44a1-b208-f004c7220c9e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645437 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645302    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645437 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645334    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645437 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645374    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f690b07a-3ac7-44a1-b208-f004c7220c9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645437 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645408    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-web-config\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645657 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645467    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645657 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645525    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f690b07a-3ac7-44a1-b208-f004c7220c9e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.645929 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.645887    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f690b07a-3ac7-44a1-b208-f004c7220c9e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.646351 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:35:09.646007    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f690b07a-3ac7-44a1-b208-f004c7220c9e-alertmanager-trusted-ca-bundle podName:f690b07a-3ac7-44a1-b208-f004c7220c9e nodeName:}" failed. No retries permitted until 2026-07-28 12:35:10.145986577 +0000 UTC m=+302.010236419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/f690b07a-3ac7-44a1-b208-f004c7220c9e-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "f690b07a-3ac7-44a1-b208-f004c7220c9e") : configmap references non-existent config key: ca-bundle.crt
Jul 28 12:35:09.648148 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.648121    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f690b07a-3ac7-44a1-b208-f004c7220c9e-config-out\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.648326 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.648303    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.648413 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.648382    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.648473 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.648413    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.648749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.648727    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.649161 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.649143    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.649334 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.649318    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.649403 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.649356    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f690b07a-3ac7-44a1-b208-f004c7220c9e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.649983 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.649966    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f690b07a-3ac7-44a1-b208-f004c7220c9e-web-config\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:09.652961 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:09.652910    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmj5d\" (UniqueName: \"kubernetes.io/projected/f690b07a-3ac7-44a1-b208-f004c7220c9e-kube-api-access-wmj5d\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:10.149825 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:10.149785    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f690b07a-3ac7-44a1-b208-f004c7220c9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:10.150684 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:10.150659    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f690b07a-3ac7-44a1-b208-f004c7220c9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f690b07a-3ac7-44a1-b208-f004c7220c9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:10.409173 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:10.409081    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 12:35:10.624410 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:10.624340    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss" event={"ID":"c38306ff-de52-4ef3-932f-ac3b727114c3","Type":"ContainerStarted","Data":"ce2e953c699a473d720b46525738db5ad6ab410de89067a78017d999797e490b"}
Jul 28 12:35:10.635295 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:10.635266    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 12:35:10.645139 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:10.645085    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-hrfss" podStartSLOduration=1.7539010400000001 podStartE2EDuration="2.64506342s" totalImagesPullingTime="891.16238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:35:10.644657026 +0000 UTC m=+302.508906885" watchObservedRunningTime="2026-07-28 12:35:10.64506342 +0000 UTC m=+302.509313279"
Jul 28 12:35:10.777905 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:35:10.777837    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf690b07a_3ac7_44a1_b208_f004c7220c9e.slice/crio-63b776c1ab3f15493e3bb9964a3e0dd887409b58553e6c883e450e4594fc25f5 WatchSource:0}: Error finding container 63b776c1ab3f15493e3bb9964a3e0dd887409b58553e6c883e450e4594fc25f5: Status 404 returned error can't find the container with id 63b776c1ab3f15493e3bb9964a3e0dd887409b58553e6c883e450e4594fc25f5
Jul 28 12:35:10.780216 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:10.780199    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 12:35:11.631970 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:11.631285    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4" event={"ID":"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f","Type":"ContainerStarted","Data":"f78aa9359e333b2a5e0f422e3a6db1e7abc4eefc1d0603ac1eec9ea7744a475d"}
Jul 28 12:35:11.631970 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:11.631393    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4" event={"ID":"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f","Type":"ContainerStarted","Data":"3e8d5aad18e883092f4a63a40ebd3ca77ff88917d41f174ec076a41e437525cc"}
Jul 28 12:35:11.631970 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:11.631421    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4" event={"ID":"7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f","Type":"ContainerStarted","Data":"30706561f63d807328614688042cdf2d94b55fcf4d2db9691c1d65438dd00b49"}
Jul 28 12:35:11.633085 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:11.633053    2549 generic.go:356] "Generic (PLEG): container finished" podID="76761059-1f76-43f6-8e6f-36ef294eba40" containerID="514195953c872c53765d6eb93e19cd35459854c78acbacc574d5d8f05b3f335d" exitCode=0
Jul 28 12:35:11.633236 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:11.633091    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2p9n" event={"ID":"76761059-1f76-43f6-8e6f-36ef294eba40","Type":"ContainerDied","Data":"514195953c872c53765d6eb93e19cd35459854c78acbacc574d5d8f05b3f335d"}
Jul 28 12:35:11.634954 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:11.634863    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f690b07a-3ac7-44a1-b208-f004c7220c9e","Type":"ContainerStarted","Data":"63b776c1ab3f15493e3bb9964a3e0dd887409b58553e6c883e450e4594fc25f5"}
Jul 28 12:35:11.673555 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:11.673503    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-t2dw4" podStartSLOduration=2.088252466 podStartE2EDuration="3.673483205s" totalImagesPullingTime="1.585230739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:35:11.652754204 +0000 UTC m=+303.517004074" watchObservedRunningTime="2026-07-28 12:35:11.673483205 +0000 UTC m=+303.537733067"
Jul 28 12:35:12.639679 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:12.639640    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2p9n" event={"ID":"76761059-1f76-43f6-8e6f-36ef294eba40","Type":"ContainerStarted","Data":"fabdc97c5f8f2a3425e6cb8ec2fb3795b5211ad5027abc6f76c8fcb3341cefa7"}
Jul 28 12:35:12.640120 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:12.639699    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c2p9n" event={"ID":"76761059-1f76-43f6-8e6f-36ef294eba40","Type":"ContainerStarted","Data":"bbf450ff64965a63ec469ca031cd4a94e2d5d65b1fbac8c7fd780dcdecda1459"}
Jul 28 12:35:12.641030 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:12.641008    2549 generic.go:356] "Generic (PLEG): container finished" podID="f690b07a-3ac7-44a1-b208-f004c7220c9e" containerID="399a3cd0beaa8a6e4b9066ddd17ecb1eaeff22f5c9d89ebbb8d8ee28073ec336" exitCode=0
Jul 28 12:35:12.641125 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:12.641091    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f690b07a-3ac7-44a1-b208-f004c7220c9e","Type":"ContainerDied","Data":"399a3cd0beaa8a6e4b9066ddd17ecb1eaeff22f5c9d89ebbb8d8ee28073ec336"}
Jul 28 12:35:12.685520 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:12.685495    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:35:12.693423 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:12.693200    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c2p9n" podStartSLOduration=2.262081197 podStartE2EDuration="4.693182001s" totalImagesPullingTime="2.312706587s" totalInitContainerRuntime="118.394217ms" isStatefulPod=true podCreationTimestamp="2026-07-28 12:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:35:12.660273661 +0000 UTC m=+304.524523520" watchObservedRunningTime="2026-07-28 12:35:12.693182001 +0000 UTC m=+304.557431861"
Jul 28 12:35:13.187956 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.187920    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"]
Jul 28 12:35:13.190741 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.190718    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:13.195861 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.195709    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-mmxxt\""
Jul 28 12:35:13.195861 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.195745    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 12:35:13.200102 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.200039    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"]
Jul 28 12:35:13.275895 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.275865    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a676e8b0-e920-43c5-96a7-64d8f56cc4cb-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5wzr7\" (UID: \"a676e8b0-e920-43c5-96a7-64d8f56cc4cb\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:13.276038 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.275935    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9kvb\" (UniqueName: \"kubernetes.io/projected/a676e8b0-e920-43c5-96a7-64d8f56cc4cb-kube-api-access-w9kvb\") pod \"monitoring-plugin-68b7fc9574-5wzr7\" (UID: \"a676e8b0-e920-43c5-96a7-64d8f56cc4cb\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:13.377466 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.377426    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a676e8b0-e920-43c5-96a7-64d8f56cc4cb-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5wzr7\" (UID: \"a676e8b0-e920-43c5-96a7-64d8f56cc4cb\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:13.377671 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.377507    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9kvb\" (UniqueName: \"kubernetes.io/projected/a676e8b0-e920-43c5-96a7-64d8f56cc4cb-kube-api-access-w9kvb\") pod \"monitoring-plugin-68b7fc9574-5wzr7\" (UID: \"a676e8b0-e920-43c5-96a7-64d8f56cc4cb\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:13.380589 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.380557    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a676e8b0-e920-43c5-96a7-64d8f56cc4cb-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5wzr7\" (UID: \"a676e8b0-e920-43c5-96a7-64d8f56cc4cb\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:13.385965 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.385941    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9kvb\" (UniqueName: \"kubernetes.io/projected/a676e8b0-e920-43c5-96a7-64d8f56cc4cb-kube-api-access-w9kvb\") pod \"monitoring-plugin-68b7fc9574-5wzr7\" (UID: \"a676e8b0-e920-43c5-96a7-64d8f56cc4cb\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:13.502576 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.502490    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:13.934784 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:13.934493    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"]
Jul 28 12:35:13.937580 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:35:13.937553    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda676e8b0_e920_43c5_96a7_64d8f56cc4cb.slice/crio-dc20c2edb3f8abf6876ce303eadf1d500f720ee330dbab427398c3266c212e2b WatchSource:0}: Error finding container dc20c2edb3f8abf6876ce303eadf1d500f720ee330dbab427398c3266c212e2b: Status 404 returned error can't find the container with id dc20c2edb3f8abf6876ce303eadf1d500f720ee330dbab427398c3266c212e2b
Jul 28 12:35:14.653930 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:14.653878    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f690b07a-3ac7-44a1-b208-f004c7220c9e","Type":"ContainerStarted","Data":"85ac9872a6b3e0e875001f342d8b40aac2d2ea675c6f7782b1cb7c80dbd7093d"}
Jul 28 12:35:14.654104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:14.653986    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f690b07a-3ac7-44a1-b208-f004c7220c9e","Type":"ContainerStarted","Data":"cf91164ff9143519da8d3401b265af4d307844c4b72891b3021e23b3761b51f5"}
Jul 28 12:35:14.654104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:14.654045    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f690b07a-3ac7-44a1-b208-f004c7220c9e","Type":"ContainerStarted","Data":"8afa3407efeb16f149025fda0ad0dedaf3538e2a8d03a33d961530bfde6658da"}
Jul 28 12:35:14.654104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:14.654085    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f690b07a-3ac7-44a1-b208-f004c7220c9e","Type":"ContainerStarted","Data":"81d22b5e6852eac5b9930b5f3a24b6c9f00496a0c0b79133f66f60ca1a33ba1e"}
Jul 28 12:35:14.654312 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:14.654130    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f690b07a-3ac7-44a1-b208-f004c7220c9e","Type":"ContainerStarted","Data":"f4d6739422f310aaee7beb594d01314c1b952cba5d575c557893c9586e6933c1"}
Jul 28 12:35:14.655370 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:14.655341    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7" event={"ID":"a676e8b0-e920-43c5-96a7-64d8f56cc4cb","Type":"ContainerStarted","Data":"dc20c2edb3f8abf6876ce303eadf1d500f720ee330dbab427398c3266c212e2b"}
Jul 28 12:35:15.663228 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:15.663191    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f690b07a-3ac7-44a1-b208-f004c7220c9e","Type":"ContainerStarted","Data":"e75fc7fe3c758705d51ea42a6deb917fd4fd93b15acaa9e89955205810a23c5f"}
Jul 28 12:35:15.700135 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:15.700086    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.895490474 podStartE2EDuration="6.700069443s" totalImagesPullingTime="2.785458617s" totalInitContainerRuntime="19.120352ms" isStatefulPod=false podCreationTimestamp="2026-07-28 12:35:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:35:15.697334797 +0000 UTC m=+307.561584656" watchObservedRunningTime="2026-07-28 12:35:15.700069443 +0000 UTC m=+307.564319303"
Jul 28 12:35:17.671971 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:17.671938    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7" event={"ID":"a676e8b0-e920-43c5-96a7-64d8f56cc4cb","Type":"ContainerStarted","Data":"bb3bdafa9a598ea5ed17a383dcffff24a9433493ebb2b7aeffeaa9d6155ff668"}
Jul 28 12:35:17.672444 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:17.672086    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:17.676973 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:17.676946    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7"
Jul 28 12:35:17.687474 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:17.687432    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5wzr7" podStartSLOduration=1.917969321 podStartE2EDuration="4.687413726s" totalImagesPullingTime="2.769444405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:35:17.687348231 +0000 UTC m=+309.551598101" watchObservedRunningTime="2026-07-28 12:35:17.687413726 +0000 UTC m=+309.551663582"
Jul 28 12:35:38.103446 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:38.103403    2549 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" podUID="a0c3087b-df3c-4ea9-af16-0b9ef84118df" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 28 12:35:48.103266 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:48.103229    2549 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" podUID="a0c3087b-df3c-4ea9-af16-0b9ef84118df" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 28 12:35:58.103564 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:58.103525    2549 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" podUID="a0c3087b-df3c-4ea9-af16-0b9ef84118df" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 28 12:35:58.104113 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:58.103601    2549 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c"
Jul 28 12:35:58.104113 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:58.104056    2549 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"9e60d6d9fa8b4a1e9346edc47b1737401773109c61dad9522a3a63df7e53e53e"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 28 12:35:58.104113 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:58.104088    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" podUID="a0c3087b-df3c-4ea9-af16-0b9ef84118df" containerName="service-proxy" containerID="cri-o://9e60d6d9fa8b4a1e9346edc47b1737401773109c61dad9522a3a63df7e53e53e" gracePeriod=30
Jul 28 12:35:58.815261 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:58.815225    2549 generic.go:356] "Generic (PLEG): container finished" podID="a0c3087b-df3c-4ea9-af16-0b9ef84118df" containerID="9e60d6d9fa8b4a1e9346edc47b1737401773109c61dad9522a3a63df7e53e53e" exitCode=2
Jul 28 12:35:58.815420 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:58.815275    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" event={"ID":"a0c3087b-df3c-4ea9-af16-0b9ef84118df","Type":"ContainerDied","Data":"9e60d6d9fa8b4a1e9346edc47b1737401773109c61dad9522a3a63df7e53e53e"}
Jul 28 12:35:58.815420 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:35:58.815309    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54447d6b67-bfw6c" event={"ID":"a0c3087b-df3c-4ea9-af16-0b9ef84118df","Type":"ContainerStarted","Data":"2fa897fbdae85048aa3c8c0c3dded1b1ef85a6f0221e385860d9cabc426ece5f"}
Jul 28 12:36:22.426589 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:36:22.426550    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:36:22.429148 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:36:22.429124    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a25858a-ae63-4104-b1a4-33ac27e882df-metrics-certs\") pod \"network-metrics-daemon-mg9wl\" (UID: \"5a25858a-ae63-4104-b1a4-33ac27e882df\") " pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:36:22.589081 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:36:22.589049    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nm4t9\""
Jul 28 12:36:22.596449 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:36:22.596428    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mg9wl"
Jul 28 12:36:22.727427 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:36:22.727405    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mg9wl"]
Jul 28 12:36:22.729671 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:36:22.729642    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a25858a_ae63_4104_b1a4_33ac27e882df.slice/crio-1ad2960ad4a89f2f8f27984023b1f7b76135ebd502f1ac1109e11b6d914fd99c WatchSource:0}: Error finding container 1ad2960ad4a89f2f8f27984023b1f7b76135ebd502f1ac1109e11b6d914fd99c: Status 404 returned error can't find the container with id 1ad2960ad4a89f2f8f27984023b1f7b76135ebd502f1ac1109e11b6d914fd99c
Jul 28 12:36:22.900778 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:36:22.900737    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mg9wl" event={"ID":"5a25858a-ae63-4104-b1a4-33ac27e882df","Type":"ContainerStarted","Data":"1ad2960ad4a89f2f8f27984023b1f7b76135ebd502f1ac1109e11b6d914fd99c"}
Jul 28 12:36:24.909999 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:36:24.909966    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mg9wl" event={"ID":"5a25858a-ae63-4104-b1a4-33ac27e882df","Type":"ContainerStarted","Data":"75445e846de6c68a8c8e446ba249ac1b407bbd1bbe8ac3ae281ae9ef5c2d55cf"}
Jul 28 12:36:24.910382 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:36:24.910022    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mg9wl" event={"ID":"5a25858a-ae63-4104-b1a4-33ac27e882df","Type":"ContainerStarted","Data":"b57242e28734ac1d77f5283edf31305ad7e5a5e4d72c5b2647705a89c1ac9647"}
Jul 28 12:36:24.928791 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:36:24.928741    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mg9wl" podStartSLOduration=375.371705935 podStartE2EDuration="6m16.928722915s" totalImagesPullingTime="1.55701698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:36:24.92806027 +0000 UTC m=+376.792310133" watchObservedRunningTime="2026-07-28 12:36:24.928722915 +0000 UTC m=+376.792972777"
Jul 28 12:39:28.609452 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.609379    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nqmrw"]
Jul 28 12:39:28.612725 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.612702    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nqmrw"
Jul 28 12:39:28.615450 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.615425    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 12:39:28.616822 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.616804    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 12:39:28.616943 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.616804    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7tvmz\""
Jul 28 12:39:28.622520 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.622500    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nqmrw"]
Jul 28 12:39:28.642784 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.642753    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzbqn\" (UniqueName: \"kubernetes.io/projected/0fdee43f-d187-4813-9187-c7c9475a1d35-kube-api-access-gzbqn\") pod \"cert-manager-55b4d954d5-nqmrw\" (UID: \"0fdee43f-d187-4813-9187-c7c9475a1d35\") " pod="cert-manager/cert-manager-55b4d954d5-nqmrw"
Jul 28 12:39:28.642924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.642810    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fdee43f-d187-4813-9187-c7c9475a1d35-bound-sa-token\") pod \"cert-manager-55b4d954d5-nqmrw\" (UID: \"0fdee43f-d187-4813-9187-c7c9475a1d35\") " pod="cert-manager/cert-manager-55b4d954d5-nqmrw"
Jul 28 12:39:28.743546 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.743513    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzbqn\" (UniqueName: \"kubernetes.io/projected/0fdee43f-d187-4813-9187-c7c9475a1d35-kube-api-access-gzbqn\") pod \"cert-manager-55b4d954d5-nqmrw\" (UID: \"0fdee43f-d187-4813-9187-c7c9475a1d35\") " pod="cert-manager/cert-manager-55b4d954d5-nqmrw"
Jul 28 12:39:28.743757 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.743578    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fdee43f-d187-4813-9187-c7c9475a1d35-bound-sa-token\") pod \"cert-manager-55b4d954d5-nqmrw\" (UID: \"0fdee43f-d187-4813-9187-c7c9475a1d35\") " pod="cert-manager/cert-manager-55b4d954d5-nqmrw"
Jul 28 12:39:28.752424 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.752387    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fdee43f-d187-4813-9187-c7c9475a1d35-bound-sa-token\") pod \"cert-manager-55b4d954d5-nqmrw\" (UID: \"0fdee43f-d187-4813-9187-c7c9475a1d35\") " pod="cert-manager/cert-manager-55b4d954d5-nqmrw"
Jul 28 12:39:28.752554 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.752501    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzbqn\" (UniqueName: \"kubernetes.io/projected/0fdee43f-d187-4813-9187-c7c9475a1d35-kube-api-access-gzbqn\") pod \"cert-manager-55b4d954d5-nqmrw\" (UID: \"0fdee43f-d187-4813-9187-c7c9475a1d35\") " pod="cert-manager/cert-manager-55b4d954d5-nqmrw"
Jul 28 12:39:28.921679 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:28.921560    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nqmrw"
Jul 28 12:39:29.047192 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:29.047162    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nqmrw"]
Jul 28 12:39:29.050162 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:39:29.050138    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fdee43f_d187_4813_9187_c7c9475a1d35.slice/crio-e6952ac36278f13e2e0b970d93e3a9f4209f42c90713199809a1e6654c81a2ef WatchSource:0}: Error finding container e6952ac36278f13e2e0b970d93e3a9f4209f42c90713199809a1e6654c81a2ef: Status 404 returned error can't find the container with id e6952ac36278f13e2e0b970d93e3a9f4209f42c90713199809a1e6654c81a2ef
Jul 28 12:39:29.561832 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:29.561789    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nqmrw" event={"ID":"0fdee43f-d187-4813-9187-c7c9475a1d35","Type":"ContainerStarted","Data":"e6952ac36278f13e2e0b970d93e3a9f4209f42c90713199809a1e6654c81a2ef"}
Jul 28 12:39:32.576141 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:32.576108    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nqmrw" event={"ID":"0fdee43f-d187-4813-9187-c7c9475a1d35","Type":"ContainerStarted","Data":"ab58b306dc745bf3caec8688799f3cf583a3e92eebdad47c684e002cdaeef339"}
Jul 28 12:39:32.595710 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:32.595660    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-nqmrw" podStartSLOduration=1.7878370380000002 podStartE2EDuration="4.595644044s" totalImagesPullingTime="2.807807006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:39:32.594510078 +0000 UTC m=+564.458759937" watchObservedRunningTime="2026-07-28 12:39:32.595644044 +0000 UTC m=+564.459893894"
Jul 28 12:39:36.412676 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.412633    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"]
Jul 28 12:39:36.414925 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.414902    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"
Jul 28 12:39:36.417856 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.417835    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 12:39:36.417951 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.417891    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 12:39:36.419428 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.419408    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cgkfj\""
Jul 28 12:39:36.423980 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.423958    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"]
Jul 28 12:39:36.498538 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.498510    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f938f96-5882-4431-9e61-ec95948a57b5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-v2bp4\" (UID: \"1f938f96-5882-4431-9e61-ec95948a57b5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"
Jul 28 12:39:36.498667 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.498598    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g58jj\" (UniqueName: \"kubernetes.io/projected/1f938f96-5882-4431-9e61-ec95948a57b5-kube-api-access-g58jj\") pod \"openshift-lws-operator-fd8ccff4c-v2bp4\" (UID: \"1f938f96-5882-4431-9e61-ec95948a57b5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"
Jul 28 12:39:36.599105 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.599059    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g58jj\" (UniqueName: \"kubernetes.io/projected/1f938f96-5882-4431-9e61-ec95948a57b5-kube-api-access-g58jj\") pod \"openshift-lws-operator-fd8ccff4c-v2bp4\" (UID: \"1f938f96-5882-4431-9e61-ec95948a57b5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"
Jul 28 12:39:36.599105 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.599118    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f938f96-5882-4431-9e61-ec95948a57b5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-v2bp4\" (UID: \"1f938f96-5882-4431-9e61-ec95948a57b5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"
Jul 28 12:39:36.599447 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.599431    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f938f96-5882-4431-9e61-ec95948a57b5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-v2bp4\" (UID: \"1f938f96-5882-4431-9e61-ec95948a57b5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"
Jul 28 12:39:36.608189 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.608157    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g58jj\" (UniqueName: \"kubernetes.io/projected/1f938f96-5882-4431-9e61-ec95948a57b5-kube-api-access-g58jj\") pod \"openshift-lws-operator-fd8ccff4c-v2bp4\" (UID: \"1f938f96-5882-4431-9e61-ec95948a57b5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"
Jul 28 12:39:36.724126 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.724043    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"
Jul 28 12:39:36.847924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:36.847897    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4"]
Jul 28 12:39:36.851421 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:39:36.851391    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f938f96_5882_4431_9e61_ec95948a57b5.slice/crio-012401295e190aa761f0c7dd2acb2a7573f91d37d4516fecf0a5978623989a9a WatchSource:0}: Error finding container 012401295e190aa761f0c7dd2acb2a7573f91d37d4516fecf0a5978623989a9a: Status 404 returned error can't find the container with id 012401295e190aa761f0c7dd2acb2a7573f91d37d4516fecf0a5978623989a9a
Jul 28 12:39:37.600335 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:37.600279    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4" event={"ID":"1f938f96-5882-4431-9e61-ec95948a57b5","Type":"ContainerStarted","Data":"012401295e190aa761f0c7dd2acb2a7573f91d37d4516fecf0a5978623989a9a"}
Jul 28 12:39:39.611085 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:39.611046    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4" event={"ID":"1f938f96-5882-4431-9e61-ec95948a57b5","Type":"ContainerStarted","Data":"e6f4cf4708fcfa4199ab88b032b1027d5c619bfbd8dab111f76e01170c4383f3"}
Jul 28 12:39:39.627961 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:39.627915    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v2bp4" podStartSLOduration=1.451403022 podStartE2EDuration="3.627895983s" totalImagesPullingTime="2.176492961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:39:39.626921084 +0000 UTC m=+571.491170945" watchObservedRunningTime="2026-07-28 12:39:39.627895983 +0000 UTC m=+571.492145846"
Jul 28 12:39:59.374338 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.374305    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"]
Jul 28 12:39:59.377398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.377374    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.381000 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.380969    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fv4tk\""
Jul 28 12:39:59.381221 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.381202    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 12:39:59.381391 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.381375    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 12:39:59.381575 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.381559    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 12:39:59.382828 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.382808    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 12:39:59.394504 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.394471    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"]
Jul 28 12:39:59.487011 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.486924    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b7043db-c8ad-4133-9d65-69ea83a34fe5-manifests\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.487011 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.486982    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb6ln\" (UniqueName: \"kubernetes.io/projected/7b7043db-c8ad-4133-9d65-69ea83a34fe5-kube-api-access-sb6ln\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.487263 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.487072    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b7043db-c8ad-4133-9d65-69ea83a34fe5-tmp\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.487263 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.487128    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b7043db-c8ad-4133-9d65-69ea83a34fe5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.487263 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.487173    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b7043db-c8ad-4133-9d65-69ea83a34fe5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.587987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.587954    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb6ln\" (UniqueName: \"kubernetes.io/projected/7b7043db-c8ad-4133-9d65-69ea83a34fe5-kube-api-access-sb6ln\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.587987 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.587993    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b7043db-c8ad-4133-9d65-69ea83a34fe5-tmp\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.588224 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.588024    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b7043db-c8ad-4133-9d65-69ea83a34fe5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.588224 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.588049    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b7043db-c8ad-4133-9d65-69ea83a34fe5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.588224 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.588091    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b7043db-c8ad-4133-9d65-69ea83a34fe5-manifests\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.588406 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.588388    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b7043db-c8ad-4133-9d65-69ea83a34fe5-manifests\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.590432 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.590405    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b7043db-c8ad-4133-9d65-69ea83a34fe5-tmp\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.590594 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.590571    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b7043db-c8ad-4133-9d65-69ea83a34fe5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.590689 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.590675    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b7043db-c8ad-4133-9d65-69ea83a34fe5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.597322 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.597300    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb6ln\" (UniqueName: \"kubernetes.io/projected/7b7043db-c8ad-4133-9d65-69ea83a34fe5-kube-api-access-sb6ln\") pod \"opendatahub-operator-controller-manager-7974886bdb-2rkn7\" (UID: \"7b7043db-c8ad-4133-9d65-69ea83a34fe5\") " pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.689645 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.689542    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:39:59.826019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:39:59.825993    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"]
Jul 28 12:39:59.829415 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:39:59.829390    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7043db_c8ad_4133_9d65_69ea83a34fe5.slice/crio-ce061d1a57c89e4436264e9e63a04e6936d101a611557f2959065d9bad24abe6 WatchSource:0}: Error finding container ce061d1a57c89e4436264e9e63a04e6936d101a611557f2959065d9bad24abe6: Status 404 returned error can't find the container with id ce061d1a57c89e4436264e9e63a04e6936d101a611557f2959065d9bad24abe6
Jul 28 12:40:00.693415 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:00.693366    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7" event={"ID":"7b7043db-c8ad-4133-9d65-69ea83a34fe5","Type":"ContainerStarted","Data":"ce061d1a57c89e4436264e9e63a04e6936d101a611557f2959065d9bad24abe6"}
Jul 28 12:40:03.703045 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:03.703004    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7" event={"ID":"7b7043db-c8ad-4133-9d65-69ea83a34fe5","Type":"ContainerStarted","Data":"e14e31ea4b86c1e3ee973986dd699296396c28bc26b593084695f1a42372571d"}
Jul 28 12:40:04.708461 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:04.708426    2549 generic.go:356] "Generic (PLEG): container finished" podID="7b7043db-c8ad-4133-9d65-69ea83a34fe5" containerID="e14e31ea4b86c1e3ee973986dd699296396c28bc26b593084695f1a42372571d" exitCode=0
Jul 28 12:40:04.708856 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:04.708466    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7" event={"ID":"7b7043db-c8ad-4133-9d65-69ea83a34fe5","Type":"ContainerDied","Data":"e14e31ea4b86c1e3ee973986dd699296396c28bc26b593084695f1a42372571d"}
Jul 28 12:40:05.715216 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:05.715182    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7" event={"ID":"7b7043db-c8ad-4133-9d65-69ea83a34fe5","Type":"ContainerStarted","Data":"69e0b798a542891b1aed8a153ae2fbedfbf113c73d24d111ea61c5ad43d7468f"}
Jul 28 12:40:05.715598 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:05.715398    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:40:05.739327 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:05.739285    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7" podStartSLOduration=2.091930453 podStartE2EDuration="6.739269687s" totalImagesPullingTime="3.584194314s" totalInitContainerRuntime="1.06314492s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:39:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:40:05.738349675 +0000 UTC m=+597.602599534" watchObservedRunningTime="2026-07-28 12:40:05.739269687 +0000 UTC m=+597.603519549"
Jul 28 12:40:08.551886 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:08.551853    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:40:08.552298 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:08.552149    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:40:16.722260 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:16.722221    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7974886bdb-2rkn7"
Jul 28 12:40:19.412889 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.412859    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b958f484-9pbr5"]
Jul 28 12:40:19.415159 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.415137    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.420141 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.420120    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 12:40:19.420400 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.420384    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 12:40:19.420993 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.420973    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 12:40:19.420993 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.420986    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 12:40:19.421135 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.421011    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9btbb\""
Jul 28 12:40:19.440510 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.440455    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b958f484-9pbr5"]
Jul 28 12:40:19.455383 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.455357    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vhxh\" (UniqueName: \"kubernetes.io/projected/6187130a-311c-4bf7-88e1-18caab96d7e7-kube-api-access-9vhxh\") pod \"kube-auth-proxy-b958f484-9pbr5\" (UID: \"6187130a-311c-4bf7-88e1-18caab96d7e7\") " pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.455468 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.455392    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6187130a-311c-4bf7-88e1-18caab96d7e7-tls-certs\") pod \"kube-auth-proxy-b958f484-9pbr5\" (UID: \"6187130a-311c-4bf7-88e1-18caab96d7e7\") " pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.455468 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.455425    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6187130a-311c-4bf7-88e1-18caab96d7e7-tmp\") pod \"kube-auth-proxy-b958f484-9pbr5\" (UID: \"6187130a-311c-4bf7-88e1-18caab96d7e7\") " pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.556360 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.556317    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vhxh\" (UniqueName: \"kubernetes.io/projected/6187130a-311c-4bf7-88e1-18caab96d7e7-kube-api-access-9vhxh\") pod \"kube-auth-proxy-b958f484-9pbr5\" (UID: \"6187130a-311c-4bf7-88e1-18caab96d7e7\") " pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.556532 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.556373    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6187130a-311c-4bf7-88e1-18caab96d7e7-tls-certs\") pod \"kube-auth-proxy-b958f484-9pbr5\" (UID: \"6187130a-311c-4bf7-88e1-18caab96d7e7\") " pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.556532 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.556426    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6187130a-311c-4bf7-88e1-18caab96d7e7-tmp\") pod \"kube-auth-proxy-b958f484-9pbr5\" (UID: \"6187130a-311c-4bf7-88e1-18caab96d7e7\") " pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.558888 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.558857    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6187130a-311c-4bf7-88e1-18caab96d7e7-tmp\") pod \"kube-auth-proxy-b958f484-9pbr5\" (UID: \"6187130a-311c-4bf7-88e1-18caab96d7e7\") " pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.559082 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.559063    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6187130a-311c-4bf7-88e1-18caab96d7e7-tls-certs\") pod \"kube-auth-proxy-b958f484-9pbr5\" (UID: \"6187130a-311c-4bf7-88e1-18caab96d7e7\") " pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.566829 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.566801    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vhxh\" (UniqueName: \"kubernetes.io/projected/6187130a-311c-4bf7-88e1-18caab96d7e7-kube-api-access-9vhxh\") pod \"kube-auth-proxy-b958f484-9pbr5\" (UID: \"6187130a-311c-4bf7-88e1-18caab96d7e7\") " pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.700955 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.700865    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cbdrp"]
Jul 28 12:40:19.703694 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.703679    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:19.706648 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.706615    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-b4ksr\""
Jul 28 12:40:19.707420 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.707402    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 12:40:19.720412 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.720387    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cbdrp"]
Jul 28 12:40:19.724907 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.724885    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5"
Jul 28 12:40:19.758250 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.758206    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c6e1025-9697-41d3-a393-b3d8e84b5098-cert\") pod \"odh-model-controller-7998c67dc4-cbdrp\" (UID: \"9c6e1025-9697-41d3-a393-b3d8e84b5098\") " pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:19.758484 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.758303    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbj2k\" (UniqueName: \"kubernetes.io/projected/9c6e1025-9697-41d3-a393-b3d8e84b5098-kube-api-access-pbj2k\") pod \"odh-model-controller-7998c67dc4-cbdrp\" (UID: \"9c6e1025-9697-41d3-a393-b3d8e84b5098\") " pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:19.859273 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.859237    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c6e1025-9697-41d3-a393-b3d8e84b5098-cert\") pod \"odh-model-controller-7998c67dc4-cbdrp\" (UID: \"9c6e1025-9697-41d3-a393-b3d8e84b5098\") " pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:19.859442 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.859303    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbj2k\" (UniqueName: \"kubernetes.io/projected/9c6e1025-9697-41d3-a393-b3d8e84b5098-kube-api-access-pbj2k\") pod \"odh-model-controller-7998c67dc4-cbdrp\" (UID: \"9c6e1025-9697-41d3-a393-b3d8e84b5098\") " pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:19.859442 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:40:19.859387    2549 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 12:40:19.859521 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:40:19.859472    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c6e1025-9697-41d3-a393-b3d8e84b5098-cert podName:9c6e1025-9697-41d3-a393-b3d8e84b5098 nodeName:}" failed. No retries permitted until 2026-07-28 12:40:20.359451043 +0000 UTC m=+612.223700880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9c6e1025-9697-41d3-a393-b3d8e84b5098-cert") pod "odh-model-controller-7998c67dc4-cbdrp" (UID: "9c6e1025-9697-41d3-a393-b3d8e84b5098") : secret "odh-model-controller-webhook-cert" not found
Jul 28 12:40:19.869424 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.869399    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b958f484-9pbr5"]
Jul 28 12:40:19.872417 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:40:19.872393    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6187130a_311c_4bf7_88e1_18caab96d7e7.slice/crio-3ccfcf3caa751c9d514d4617eaf596104c6b4c8386ccfcfc77ba316397bcbfba WatchSource:0}: Error finding container 3ccfcf3caa751c9d514d4617eaf596104c6b4c8386ccfcfc77ba316397bcbfba: Status 404 returned error can't find the container with id 3ccfcf3caa751c9d514d4617eaf596104c6b4c8386ccfcfc77ba316397bcbfba
Jul 28 12:40:19.874197 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.874176    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbj2k\" (UniqueName: \"kubernetes.io/projected/9c6e1025-9697-41d3-a393-b3d8e84b5098-kube-api-access-pbj2k\") pod \"odh-model-controller-7998c67dc4-cbdrp\" (UID: \"9c6e1025-9697-41d3-a393-b3d8e84b5098\") " pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:19.874318 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:19.874302    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 12:40:20.364611 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.364568    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c6e1025-9697-41d3-a393-b3d8e84b5098-cert\") pod \"odh-model-controller-7998c67dc4-cbdrp\" (UID: \"9c6e1025-9697-41d3-a393-b3d8e84b5098\") " pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:20.367294 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.367271    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c6e1025-9697-41d3-a393-b3d8e84b5098-cert\") pod \"odh-model-controller-7998c67dc4-cbdrp\" (UID: \"9c6e1025-9697-41d3-a393-b3d8e84b5098\") " pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:20.613884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.613846    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:20.645856 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.645820    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f26sg"]
Jul 28 12:40:20.648880 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.648854    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:20.653146 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.653123    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 12:40:20.653490 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.653466    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-gmx9r\""
Jul 28 12:40:20.661307 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.660934    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f26sg"]
Jul 28 12:40:20.667849 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.667823    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f12ed3b-d534-49ed-996b-d079bfaa4d0d-tls-certs\") pod \"model-serving-api-54d89bd5fc-f26sg\" (UID: \"0f12ed3b-d534-49ed-996b-d079bfaa4d0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:20.667972 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.667880    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfqbm\" (UniqueName: \"kubernetes.io/projected/0f12ed3b-d534-49ed-996b-d079bfaa4d0d-kube-api-access-qfqbm\") pod \"model-serving-api-54d89bd5fc-f26sg\" (UID: \"0f12ed3b-d534-49ed-996b-d079bfaa4d0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:20.769742 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.769704    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f12ed3b-d534-49ed-996b-d079bfaa4d0d-tls-certs\") pod \"model-serving-api-54d89bd5fc-f26sg\" (UID: \"0f12ed3b-d534-49ed-996b-d079bfaa4d0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:20.769919 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:40:20.769841    2549 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 28 12:40:20.769919 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:40:20.769909    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f12ed3b-d534-49ed-996b-d079bfaa4d0d-tls-certs podName:0f12ed3b-d534-49ed-996b-d079bfaa4d0d nodeName:}" failed. No retries permitted until 2026-07-28 12:40:21.269885848 +0000 UTC m=+613.134135692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0f12ed3b-d534-49ed-996b-d079bfaa4d0d-tls-certs") pod "model-serving-api-54d89bd5fc-f26sg" (UID: "0f12ed3b-d534-49ed-996b-d079bfaa4d0d") : secret "model-serving-api-tls" not found
Jul 28 12:40:20.770117 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.770090    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqbm\" (UniqueName: \"kubernetes.io/projected/0f12ed3b-d534-49ed-996b-d079bfaa4d0d-kube-api-access-qfqbm\") pod \"model-serving-api-54d89bd5fc-f26sg\" (UID: \"0f12ed3b-d534-49ed-996b-d079bfaa4d0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:20.771294 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.771272    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cbdrp"]
Jul 28 12:40:20.773950 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:40:20.773883    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c6e1025_9697_41d3_a393_b3d8e84b5098.slice/crio-55b524618173d9447aa4d85a2214fc13dd2e5c61311310f5ed613e0ca9edc7ce WatchSource:0}: Error finding container 55b524618173d9447aa4d85a2214fc13dd2e5c61311310f5ed613e0ca9edc7ce: Status 404 returned error can't find the container with id 55b524618173d9447aa4d85a2214fc13dd2e5c61311310f5ed613e0ca9edc7ce
Jul 28 12:40:20.777050 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.777021    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5" event={"ID":"6187130a-311c-4bf7-88e1-18caab96d7e7","Type":"ContainerStarted","Data":"3ccfcf3caa751c9d514d4617eaf596104c6b4c8386ccfcfc77ba316397bcbfba"}
Jul 28 12:40:20.784334 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.784307    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfqbm\" (UniqueName: \"kubernetes.io/projected/0f12ed3b-d534-49ed-996b-d079bfaa4d0d-kube-api-access-qfqbm\") pod \"model-serving-api-54d89bd5fc-f26sg\" (UID: \"0f12ed3b-d534-49ed-996b-d079bfaa4d0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:20.874710 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.874613    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"]
Jul 28 12:40:20.877796 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.877774    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:20.880912 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.880754    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 12:40:20.880912 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.880767    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 12:40:20.880912 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.880754    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 12:40:20.880912 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.880790    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 12:40:20.880912 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.880840    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h6vl8\""
Jul 28 12:40:20.889666 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.889588    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"]
Jul 28 12:40:20.971757 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.971729    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:20.971928 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.971773    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cpfn\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-kube-api-access-2cpfn\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:20.971928 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.971794    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:20.971928 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.971810    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:20.971928 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.971898    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:20.972077 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.971928    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59a5c96d-049d-43ae-8bc1-f61d2681aad6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:20.972077 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:20.971947    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.073081 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.073046    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.073081 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.073081    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.073338 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.073124    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.073338 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.073153    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59a5c96d-049d-43ae-8bc1-f61d2681aad6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.073338 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.073170    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.073338 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.073203    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.073338 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.073251    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cpfn\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-kube-api-access-2cpfn\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.073843 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.073811    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.075891 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.075867    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.075891 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.075877    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59a5c96d-049d-43ae-8bc1-f61d2681aad6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.076027 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.075872    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.076027 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.075908    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.081839 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.081814    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cpfn\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-kube-api-access-2cpfn\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.082221 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.082202    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r2crp\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.191086 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.191006    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:21.275451 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.275411    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f12ed3b-d534-49ed-996b-d079bfaa4d0d-tls-certs\") pod \"model-serving-api-54d89bd5fc-f26sg\" (UID: \"0f12ed3b-d534-49ed-996b-d079bfaa4d0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:21.278219 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.278192    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f12ed3b-d534-49ed-996b-d079bfaa4d0d-tls-certs\") pod \"model-serving-api-54d89bd5fc-f26sg\" (UID: \"0f12ed3b-d534-49ed-996b-d079bfaa4d0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:21.337071 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.337039    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"]
Jul 28 12:40:21.340270 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:40:21.340238    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a5c96d_049d_43ae_8bc1_f61d2681aad6.slice/crio-fa9a01fc407ba6993938da2274ed2e80c5906d6fa97c96ee662e581005db2906 WatchSource:0}: Error finding container fa9a01fc407ba6993938da2274ed2e80c5906d6fa97c96ee662e581005db2906: Status 404 returned error can't find the container with id fa9a01fc407ba6993938da2274ed2e80c5906d6fa97c96ee662e581005db2906
Jul 28 12:40:21.568126 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.568092    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:21.746137 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.746065    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f26sg"]
Jul 28 12:40:21.748498 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:40:21.748444    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f12ed3b_d534_49ed_996b_d079bfaa4d0d.slice/crio-e4920198b08523dd677f8082100dc70394aeb9d9b474adb9288a8bd17799e70b WatchSource:0}: Error finding container e4920198b08523dd677f8082100dc70394aeb9d9b474adb9288a8bd17799e70b: Status 404 returned error can't find the container with id e4920198b08523dd677f8082100dc70394aeb9d9b474adb9288a8bd17799e70b
Jul 28 12:40:21.783730 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.783696    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp" event={"ID":"9c6e1025-9697-41d3-a393-b3d8e84b5098","Type":"ContainerStarted","Data":"55b524618173d9447aa4d85a2214fc13dd2e5c61311310f5ed613e0ca9edc7ce"}
Jul 28 12:40:21.785445 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.785407    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-f26sg" event={"ID":"0f12ed3b-d534-49ed-996b-d079bfaa4d0d","Type":"ContainerStarted","Data":"e4920198b08523dd677f8082100dc70394aeb9d9b474adb9288a8bd17799e70b"}
Jul 28 12:40:21.787292 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:21.787263    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp" event={"ID":"59a5c96d-049d-43ae-8bc1-f61d2681aad6","Type":"ContainerStarted","Data":"fa9a01fc407ba6993938da2274ed2e80c5906d6fa97c96ee662e581005db2906"}
Jul 28 12:40:24.804556 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:24.804513    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5" event={"ID":"6187130a-311c-4bf7-88e1-18caab96d7e7","Type":"ContainerStarted","Data":"efbf0053982c4b189c09ab13ac22c256a20ddbdb65c611e61138db5cb104ecda"}
Jul 28 12:40:24.806290 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:24.806260    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp" event={"ID":"9c6e1025-9697-41d3-a393-b3d8e84b5098","Type":"ContainerStarted","Data":"3c7b28ff677059f5e8cf23f37647965c33087eb0f1d4e7bf66f73cc17bc9c838"}
Jul 28 12:40:24.806424 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:24.806412    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:24.827868 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:24.822848    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b958f484-9pbr5" podStartSLOduration=1.625564754 podStartE2EDuration="5.822830964s" totalImagesPullingTime="4.19726621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:40:24.822248411 +0000 UTC m=+616.686498270" watchObservedRunningTime="2026-07-28 12:40:24.822830964 +0000 UTC m=+616.687080825"
Jul 28 12:40:24.841617 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:24.841555    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp" podStartSLOduration=2.496297797 podStartE2EDuration="5.841535744s" totalImagesPullingTime="3.345237947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:40:24.841474286 +0000 UTC m=+616.705724149" watchObservedRunningTime="2026-07-28 12:40:24.841535744 +0000 UTC m=+616.705785614"
Jul 28 12:40:25.765349 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:25.765298    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:40:25.765464 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:25.765399    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:40:25.811986 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:25.811954    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-f26sg" event={"ID":"0f12ed3b-d534-49ed-996b-d079bfaa4d0d","Type":"ContainerStarted","Data":"7ded42ba32b0ec590a5caf60f61335722f14dcb4feaec3d7d46f4e424c25f35f"}
Jul 28 12:40:25.812518 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:25.812096    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:25.832962 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:25.832917    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-f26sg" podStartSLOduration=1.879096828 podStartE2EDuration="5.832901752s" totalImagesPullingTime="3.953804924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:40:25.8321113 +0000 UTC m=+617.696361160" watchObservedRunningTime="2026-07-28 12:40:25.832901752 +0000 UTC m=+617.697151611"
Jul 28 12:40:26.818387 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:26.818344    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp" event={"ID":"59a5c96d-049d-43ae-8bc1-f61d2681aad6","Type":"ContainerStarted","Data":"5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6"}
Jul 28 12:40:26.818789 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:26.818532    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:26.820143 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:26.820114    2549 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-r2crp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 12:40:26.820250 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:26.820166    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp" podUID="59a5c96d-049d-43ae-8bc1-f61d2681aad6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 12:40:26.841721 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:26.841666    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp" podStartSLOduration=2.419344544 podStartE2EDuration="6.841647172s" totalImagesPullingTime="4.422302628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:40:26.839086263 +0000 UTC m=+618.703336145" watchObservedRunningTime="2026-07-28 12:40:26.841647172 +0000 UTC m=+618.705897029"
Jul 28 12:40:27.387011 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.386971    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"]
Jul 28 12:40:27.390410 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.390378    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.393718 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.393696    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-sgsks\""
Jul 28 12:40:27.406104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.406079    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"]
Jul 28 12:40:27.430948 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.430914    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.431096 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.430957    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.431096 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.431005    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.431096 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.431047    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.431263 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.431111    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.431263 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.431142    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz29m\" (UniqueName: \"kubernetes.io/projected/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-kube-api-access-xz29m\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.431263 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.431187    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.431263 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.431225    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.431404 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.431335    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532305 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532267    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532473 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532315    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532473 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532380    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532473 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532410    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532665 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532529    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532665 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532579    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532774 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532710    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532774 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532766    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532878 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532779    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532878 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532793    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz29m\" (UniqueName: \"kubernetes.io/projected/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-kube-api-access-xz29m\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532878 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532811    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.532878 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.532839    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.533060 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.533018    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.533386 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.533319    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.534939 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.534918    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.535200 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.535180    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.560534 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.560498    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.561048 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.561021    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz29m\" (UniqueName: \"kubernetes.io/projected/bdb1d50f-0db7-4127-aa3a-65dad7bb40af-kube-api-access-xz29m\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29\" (UID: \"bdb1d50f-0db7-4127-aa3a-65dad7bb40af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.700998 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.700881    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:27.823230 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.823204    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:27.852270 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:27.852248    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"]
Jul 28 12:40:27.854042 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:40:27.854020    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb1d50f_0db7_4127_aa3a_65dad7bb40af.slice/crio-67afae7ba7452ed030f04af9fa1265a96ebc3ced56ec3bae1977abd440427e3f WatchSource:0}: Error finding container 67afae7ba7452ed030f04af9fa1265a96ebc3ced56ec3bae1977abd440427e3f: Status 404 returned error can't find the container with id 67afae7ba7452ed030f04af9fa1265a96ebc3ced56ec3bae1977abd440427e3f
Jul 28 12:40:28.827384 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:28.827297    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29" event={"ID":"bdb1d50f-0db7-4127-aa3a-65dad7bb40af","Type":"ContainerStarted","Data":"67afae7ba7452ed030f04af9fa1265a96ebc3ced56ec3bae1977abd440427e3f"}
Jul 28 12:40:29.763601 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.763572    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"]
Jul 28 12:40:29.766969 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.766950    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:40:29.771462 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.771444    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5p479\""
Jul 28 12:40:29.771797 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.771770    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 12:40:29.777844 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.777824    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"]
Jul 28 12:40:29.781201 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.781180    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:40:29.783574 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.783555    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"]
Jul 28 12:40:29.785586 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.785565    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9sxz2\""
Jul 28 12:40:29.786182 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.786153    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 12:40:29.802514 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.802481    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"]
Jul 28 12:40:29.855522 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.855496    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/033608ea-141b-49ce-a3c7-b9b5e2c43924-cert\") pod \"kserve-controller-manager-d6f4b65b8-flt4v\" (UID: \"033608ea-141b-49ce-a3c7-b9b5e2c43924\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:40:29.855889 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.855527    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsr72\" (UniqueName: \"kubernetes.io/projected/033608ea-141b-49ce-a3c7-b9b5e2c43924-kube-api-access-dsr72\") pod \"kserve-controller-manager-d6f4b65b8-flt4v\" (UID: \"033608ea-141b-49ce-a3c7-b9b5e2c43924\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:40:29.855889 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.855561    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/288e06ff-932a-4134-80a1-3fe602c1e6dd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rx4f2\" (UID: \"288e06ff-932a-4134-80a1-3fe602c1e6dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:40:29.855889 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.855678    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srj7k\" (UniqueName: \"kubernetes.io/projected/288e06ff-932a-4134-80a1-3fe602c1e6dd-kube-api-access-srj7k\") pod \"llmisvc-controller-manager-777c6bd7d5-rx4f2\" (UID: \"288e06ff-932a-4134-80a1-3fe602c1e6dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:40:29.956838 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.956807    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srj7k\" (UniqueName: \"kubernetes.io/projected/288e06ff-932a-4134-80a1-3fe602c1e6dd-kube-api-access-srj7k\") pod \"llmisvc-controller-manager-777c6bd7d5-rx4f2\" (UID: \"288e06ff-932a-4134-80a1-3fe602c1e6dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:40:29.956986 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.956853    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/033608ea-141b-49ce-a3c7-b9b5e2c43924-cert\") pod \"kserve-controller-manager-d6f4b65b8-flt4v\" (UID: \"033608ea-141b-49ce-a3c7-b9b5e2c43924\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:40:29.956986 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.956871    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsr72\" (UniqueName: \"kubernetes.io/projected/033608ea-141b-49ce-a3c7-b9b5e2c43924-kube-api-access-dsr72\") pod \"kserve-controller-manager-d6f4b65b8-flt4v\" (UID: \"033608ea-141b-49ce-a3c7-b9b5e2c43924\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:40:29.956986 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.956909    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/288e06ff-932a-4134-80a1-3fe602c1e6dd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rx4f2\" (UID: \"288e06ff-932a-4134-80a1-3fe602c1e6dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:40:29.957078 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:40:29.957019    2549 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 28 12:40:29.957078 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:40:29.957073    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/288e06ff-932a-4134-80a1-3fe602c1e6dd-cert podName:288e06ff-932a-4134-80a1-3fe602c1e6dd nodeName:}" failed. No retries permitted until 2026-07-28 12:40:30.457057891 +0000 UTC m=+622.321307731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/288e06ff-932a-4134-80a1-3fe602c1e6dd-cert") pod "llmisvc-controller-manager-777c6bd7d5-rx4f2" (UID: "288e06ff-932a-4134-80a1-3fe602c1e6dd") : secret "llmisvc-webhook-server-cert" not found
Jul 28 12:40:29.959449 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.959429    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/033608ea-141b-49ce-a3c7-b9b5e2c43924-cert\") pod \"kserve-controller-manager-d6f4b65b8-flt4v\" (UID: \"033608ea-141b-49ce-a3c7-b9b5e2c43924\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:40:29.967153 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.967121    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsr72\" (UniqueName: \"kubernetes.io/projected/033608ea-141b-49ce-a3c7-b9b5e2c43924-kube-api-access-dsr72\") pod \"kserve-controller-manager-d6f4b65b8-flt4v\" (UID: \"033608ea-141b-49ce-a3c7-b9b5e2c43924\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:40:29.967271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:29.967254    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srj7k\" (UniqueName: \"kubernetes.io/projected/288e06ff-932a-4134-80a1-3fe602c1e6dd-kube-api-access-srj7k\") pod \"llmisvc-controller-manager-777c6bd7d5-rx4f2\" (UID: \"288e06ff-932a-4134-80a1-3fe602c1e6dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:40:30.078231 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:30.078197    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:40:30.248796 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:30.248766    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"]
Jul 28 12:40:30.253886 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:40:30.253861    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033608ea_141b_49ce_a3c7_b9b5e2c43924.slice/crio-2a3d4e4a6b9b33342a94a47b09cae7ce8e43c1f6581b298d62a1a83dbf92c0de WatchSource:0}: Error finding container 2a3d4e4a6b9b33342a94a47b09cae7ce8e43c1f6581b298d62a1a83dbf92c0de: Status 404 returned error can't find the container with id 2a3d4e4a6b9b33342a94a47b09cae7ce8e43c1f6581b298d62a1a83dbf92c0de
Jul 28 12:40:30.462364 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:30.462277    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/288e06ff-932a-4134-80a1-3fe602c1e6dd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rx4f2\" (UID: \"288e06ff-932a-4134-80a1-3fe602c1e6dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:40:30.465333 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:30.465306    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/288e06ff-932a-4134-80a1-3fe602c1e6dd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rx4f2\" (UID: \"288e06ff-932a-4134-80a1-3fe602c1e6dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:40:30.691584 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:30.691548    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:40:30.840018 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:30.839980    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v" event={"ID":"033608ea-141b-49ce-a3c7-b9b5e2c43924","Type":"ContainerStarted","Data":"2a3d4e4a6b9b33342a94a47b09cae7ce8e43c1f6581b298d62a1a83dbf92c0de"}
Jul 28 12:40:30.873197 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:30.873167    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"]
Jul 28 12:40:30.875717 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:40:30.875687    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod288e06ff_932a_4134_80a1_3fe602c1e6dd.slice/crio-2ed27dbdc3cca0a0ea0e84890727ce3f00402c2190997829c9c3cc31f8c8e7ce WatchSource:0}: Error finding container 2ed27dbdc3cca0a0ea0e84890727ce3f00402c2190997829c9c3cc31f8c8e7ce: Status 404 returned error can't find the container with id 2ed27dbdc3cca0a0ea0e84890727ce3f00402c2190997829c9c3cc31f8c8e7ce
Jul 28 12:40:31.713953 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:31.713923    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"]
Jul 28 12:40:31.714127 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:31.714101    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp" podUID="59a5c96d-049d-43ae-8bc1-f61d2681aad6" containerName="discovery" containerID="cri-o://5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6" gracePeriod=30
Jul 28 12:40:31.848303 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:31.848261    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2" event={"ID":"288e06ff-932a-4134-80a1-3fe602c1e6dd","Type":"ContainerStarted","Data":"2ed27dbdc3cca0a0ea0e84890727ce3f00402c2190997829c9c3cc31f8c8e7ce"}
Jul 28 12:40:32.229662 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.229489    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:40:32.229662 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.229558    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:40:32.229662 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.229588    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:40:32.834361 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.834329    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:32.854429 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.854394    2549 generic.go:356] "Generic (PLEG): container finished" podID="59a5c96d-049d-43ae-8bc1-f61d2681aad6" containerID="5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6" exitCode=0
Jul 28 12:40:32.854600 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.854439    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"
Jul 28 12:40:32.854600 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.854483    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp" event={"ID":"59a5c96d-049d-43ae-8bc1-f61d2681aad6","Type":"ContainerDied","Data":"5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6"}
Jul 28 12:40:32.854600 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.854547    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp" event={"ID":"59a5c96d-049d-43ae-8bc1-f61d2681aad6","Type":"ContainerDied","Data":"fa9a01fc407ba6993938da2274ed2e80c5906d6fa97c96ee662e581005db2906"}
Jul 28 12:40:32.854803 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.854590    2549 scope.go:117] "RemoveContainer" containerID="5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6"
Jul 28 12:40:32.856035 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.856014    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29" event={"ID":"bdb1d50f-0db7-4127-aa3a-65dad7bb40af","Type":"ContainerStarted","Data":"d300e633d090d5627fd9c1f4813004ad8499e6a591a1dd5709f82a5c924b64c1"}
Jul 28 12:40:32.866777 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.866761    2549 scope.go:117] "RemoveContainer" containerID="5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6"
Jul 28 12:40:32.867065 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:40:32.867044    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6\": container with ID starting with 5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6 not found: ID does not exist" containerID="5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6"
Jul 28 12:40:32.867140 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.867070    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6"} err="failed to get container status \"5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6\": rpc error: code = NotFound desc = could not find container \"5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6\": container with ID starting with 5b0d9f2fe6ba1bfb4bc34eea61f09385e7ad65fec010e3685d892c09993ba3f6 not found: ID does not exist"
Jul 28 12:40:32.884826 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.884782    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29" podStartSLOduration=1.511532495 podStartE2EDuration="5.884764587s" totalImagesPullingTime="4.373232092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:40:32.883214631 +0000 UTC m=+624.747464491" watchObservedRunningTime="2026-07-28 12:40:32.884764587 +0000 UTC m=+624.749014445"
Jul 28 12:40:32.887543 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.887524    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cpfn\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-kube-api-access-2cpfn\") pod \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") "
Jul 28 12:40:32.887667 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.887636    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-cacerts\") pod \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") "
Jul 28 12:40:32.887739 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.887670    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-ca-configmap\") pod \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") "
Jul 28 12:40:32.887739 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.887696    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-dns-cert\") pod \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") "
Jul 28 12:40:32.887739 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.887735    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-token\") pod \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") "
Jul 28 12:40:32.887913 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.887796    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-kubeconfig\") pod \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") "
Jul 28 12:40:32.887913 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.887821    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59a5c96d-049d-43ae-8bc1-f61d2681aad6-local-certs\") pod \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\" (UID: \"59a5c96d-049d-43ae-8bc1-f61d2681aad6\") "
Jul 28 12:40:32.888546 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.888436    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "59a5c96d-049d-43ae-8bc1-f61d2681aad6" (UID: "59a5c96d-049d-43ae-8bc1-f61d2681aad6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:40:32.890680 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.890643    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-cacerts" (OuterVolumeSpecName: "cacerts") pod "59a5c96d-049d-43ae-8bc1-f61d2681aad6" (UID: "59a5c96d-049d-43ae-8bc1-f61d2681aad6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:40:32.890887 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.890864    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59a5c96d-049d-43ae-8bc1-f61d2681aad6-local-certs" (OuterVolumeSpecName: "local-certs") pod "59a5c96d-049d-43ae-8bc1-f61d2681aad6" (UID: "59a5c96d-049d-43ae-8bc1-f61d2681aad6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:40:32.891100 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.891081    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-kube-api-access-2cpfn" (OuterVolumeSpecName: "kube-api-access-2cpfn") pod "59a5c96d-049d-43ae-8bc1-f61d2681aad6" (UID: "59a5c96d-049d-43ae-8bc1-f61d2681aad6"). InnerVolumeSpecName "kube-api-access-2cpfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:40:32.891100 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.891085    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "59a5c96d-049d-43ae-8bc1-f61d2681aad6" (UID: "59a5c96d-049d-43ae-8bc1-f61d2681aad6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:40:32.891230 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.891128    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "59a5c96d-049d-43ae-8bc1-f61d2681aad6" (UID: "59a5c96d-049d-43ae-8bc1-f61d2681aad6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:40:32.891413 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.891395    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-token" (OuterVolumeSpecName: "istio-token") pod "59a5c96d-049d-43ae-8bc1-f61d2681aad6" (UID: "59a5c96d-049d-43ae-8bc1-f61d2681aad6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:40:32.991346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.989665    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cpfn\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-kube-api-access-2cpfn\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:40:32.991346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.989702    2549 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-cacerts\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:40:32.991346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.989729    2549 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-ca-configmap\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:40:32.991346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.989746    2549 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-csr-dns-cert\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:40:32.991346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.989762    2549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-token\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:40:32.991346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.989777    2549 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59a5c96d-049d-43ae-8bc1-f61d2681aad6-istio-kubeconfig\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:40:32.991346 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:32.989797    2549 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59a5c96d-049d-43ae-8bc1-f61d2681aad6-local-certs\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:40:33.201938 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:33.201907    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"]
Jul 28 12:40:33.207365 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:33.207337    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2crp"]
Jul 28 12:40:33.702235 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:33.702192    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:33.707968 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:33.707928    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:33.862479 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:33.862442    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v" event={"ID":"033608ea-141b-49ce-a3c7-b9b5e2c43924","Type":"ContainerStarted","Data":"52a0d5768049df9ffa0b81e0af5c32e191179b4fda1ad6e49845b726aadfe60a"}
Jul 28 12:40:33.862656 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:33.862544    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:40:33.863751 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:33.863732    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:33.864562 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:33.864549    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29"
Jul 28 12:40:33.879095 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:33.879055    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v" podStartSLOduration=2.26484598 podStartE2EDuration="4.879042273s" totalImagesPullingTime="2.614196293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:40:33.877973147 +0000 UTC m=+625.742223007" watchObservedRunningTime="2026-07-28 12:40:33.879042273 +0000 UTC m=+625.743292131"
Jul 28 12:40:34.693078 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:34.693042    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59a5c96d-049d-43ae-8bc1-f61d2681aad6" path="/var/lib/kubelet/pods/59a5c96d-049d-43ae-8bc1-f61d2681aad6/volumes"
Jul 28 12:40:35.815174 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:35.815147    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-cbdrp"
Jul 28 12:40:35.875993 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:35.875913    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2" event={"ID":"288e06ff-932a-4134-80a1-3fe602c1e6dd","Type":"ContainerStarted","Data":"da2d81aecbaf87d22acd50470fe88f09c9f9ebc0673b2fc209ea2721a7a70965"}
Jul 28 12:40:35.894172 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:35.894125    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2" podStartSLOduration=2.334770755 podStartE2EDuration="6.894111415s" totalImagesPullingTime="4.55934066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:40:35.893747453 +0000 UTC m=+627.757997354" watchObservedRunningTime="2026-07-28 12:40:35.894111415 +0000 UTC m=+627.758361274"
Jul 28 12:40:36.823097 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:36.823062    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-f26sg"
Jul 28 12:40:36.880711 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:40:36.880685    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:41:04.874171 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:04.874139    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-flt4v"
Jul 28 12:41:07.887285 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:07.887254    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rx4f2"
Jul 28 12:41:20.613637 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.613584    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2jzpt"]
Jul 28 12:41:20.614110 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.614080    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59a5c96d-049d-43ae-8bc1-f61d2681aad6" containerName="discovery"
Jul 28 12:41:20.614110 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.614099    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="59a5c96d-049d-43ae-8bc1-f61d2681aad6" containerName="discovery"
Jul 28 12:41:20.614257 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.614229    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="59a5c96d-049d-43ae-8bc1-f61d2681aad6" containerName="discovery"
Jul 28 12:41:20.622274 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.622241    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt"
Jul 28 12:41:20.625986 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.625781    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-8gr7b\""
Jul 28 12:41:20.625986 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.625848    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 12:41:20.626224 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.626200    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2jzpt"]
Jul 28 12:41:20.626355 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.626295    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 12:41:20.703389 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.703359    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76rr5\" (UniqueName: \"kubernetes.io/projected/0bcb6b45-03e0-42f6-a055-55e9bd273c6e-kube-api-access-76rr5\") pod \"kuadrant-operator-catalog-2jzpt\" (UID: \"0bcb6b45-03e0-42f6-a055-55e9bd273c6e\") " pod="kuadrant-system/kuadrant-operator-catalog-2jzpt"
Jul 28 12:41:20.804543 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.804509    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76rr5\" (UniqueName: \"kubernetes.io/projected/0bcb6b45-03e0-42f6-a055-55e9bd273c6e-kube-api-access-76rr5\") pod \"kuadrant-operator-catalog-2jzpt\" (UID: \"0bcb6b45-03e0-42f6-a055-55e9bd273c6e\") " pod="kuadrant-system/kuadrant-operator-catalog-2jzpt"
Jul 28 12:41:20.813036 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.813007    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76rr5\" (UniqueName: \"kubernetes.io/projected/0bcb6b45-03e0-42f6-a055-55e9bd273c6e-kube-api-access-76rr5\") pod \"kuadrant-operator-catalog-2jzpt\" (UID: \"0bcb6b45-03e0-42f6-a055-55e9bd273c6e\") " pod="kuadrant-system/kuadrant-operator-catalog-2jzpt"
Jul 28 12:41:20.933981 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.933887    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt"
Jul 28 12:41:20.985995 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:20.985962    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2jzpt"]
Jul 28 12:41:21.059874 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.059847    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2jzpt"]
Jul 28 12:41:21.063267 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:41:21.063238    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bcb6b45_03e0_42f6_a055_55e9bd273c6e.slice/crio-d359f7ab1c8e0d51bbcbb330613268ad7a32e3486a096a536a181229f8084aa9 WatchSource:0}: Error finding container d359f7ab1c8e0d51bbcbb330613268ad7a32e3486a096a536a181229f8084aa9: Status 404 returned error can't find the container with id d359f7ab1c8e0d51bbcbb330613268ad7a32e3486a096a536a181229f8084aa9
Jul 28 12:41:21.069916 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.069886    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt" event={"ID":"0bcb6b45-03e0-42f6-a055-55e9bd273c6e","Type":"ContainerStarted","Data":"d359f7ab1c8e0d51bbcbb330613268ad7a32e3486a096a536a181229f8084aa9"}
Jul 28 12:41:21.196522 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.196442    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp4jv"]
Jul 28 12:41:21.199146 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.199131    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp4jv"
Jul 28 12:41:21.211514 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.211489    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp4jv"]
Jul 28 12:41:21.309303 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.309263    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdh26\" (UniqueName: \"kubernetes.io/projected/aa2edebf-b387-42de-9b5a-a243fb3ece48-kube-api-access-tdh26\") pod \"kuadrant-operator-catalog-kp4jv\" (UID: \"aa2edebf-b387-42de-9b5a-a243fb3ece48\") " pod="kuadrant-system/kuadrant-operator-catalog-kp4jv"
Jul 28 12:41:21.410207 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.410180    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdh26\" (UniqueName: \"kubernetes.io/projected/aa2edebf-b387-42de-9b5a-a243fb3ece48-kube-api-access-tdh26\") pod \"kuadrant-operator-catalog-kp4jv\" (UID: \"aa2edebf-b387-42de-9b5a-a243fb3ece48\") " pod="kuadrant-system/kuadrant-operator-catalog-kp4jv"
Jul 28 12:41:21.418810 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.418787    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdh26\" (UniqueName: \"kubernetes.io/projected/aa2edebf-b387-42de-9b5a-a243fb3ece48-kube-api-access-tdh26\") pod \"kuadrant-operator-catalog-kp4jv\" (UID: \"aa2edebf-b387-42de-9b5a-a243fb3ece48\") " pod="kuadrant-system/kuadrant-operator-catalog-kp4jv"
Jul 28 12:41:21.508163 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.508073    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kp4jv"
Jul 28 12:41:21.655504 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:21.655480    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kp4jv"]
Jul 28 12:41:21.657705 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:41:21.657682    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2edebf_b387_42de_9b5a_a243fb3ece48.slice/crio-2ef1225004a33182b10e2f171bad82a8b7b61506269a6cf7ae959f4754a5822e WatchSource:0}: Error finding container 2ef1225004a33182b10e2f171bad82a8b7b61506269a6cf7ae959f4754a5822e: Status 404 returned error can't find the container with id 2ef1225004a33182b10e2f171bad82a8b7b61506269a6cf7ae959f4754a5822e
Jul 28 12:41:22.076204 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:22.076168    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp4jv" event={"ID":"aa2edebf-b387-42de-9b5a-a243fb3ece48","Type":"ContainerStarted","Data":"2ef1225004a33182b10e2f171bad82a8b7b61506269a6cf7ae959f4754a5822e"}
Jul 28 12:41:24.088663 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:24.088599    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kp4jv" event={"ID":"aa2edebf-b387-42de-9b5a-a243fb3ece48","Type":"ContainerStarted","Data":"4211718983daebb049783f2c1120361c5f70b10514bce3354bdd7c66787a1845"}
Jul 28 12:41:24.090120 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:24.090095    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt" event={"ID":"0bcb6b45-03e0-42f6-a055-55e9bd273c6e","Type":"ContainerStarted","Data":"4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218"}
Jul 28 12:41:24.090239 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:24.090183    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt" podUID="0bcb6b45-03e0-42f6-a055-55e9bd273c6e" containerName="registry-server" containerID="cri-o://4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218" gracePeriod=2
Jul 28 12:41:24.106443 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:24.106405    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kp4jv" podStartSLOduration=1.382367312 podStartE2EDuration="3.106390857s" totalImagesPullingTime="1.724023545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:41:24.105005861 +0000 UTC m=+675.969255721" watchObservedRunningTime="2026-07-28 12:41:24.106390857 +0000 UTC m=+675.970640716"
Jul 28 12:41:24.121810 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:24.121761    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt" podStartSLOduration=1.807026364 podStartE2EDuration="4.121744169s" totalImagesPullingTime="2.314717805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:41:24.120514953 +0000 UTC m=+675.984764812" watchObservedRunningTime="2026-07-28 12:41:24.121744169 +0000 UTC m=+675.985994027"
Jul 28 12:41:24.328568 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:24.328545    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt"
Jul 28 12:41:24.437311 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:24.437220    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76rr5\" (UniqueName: \"kubernetes.io/projected/0bcb6b45-03e0-42f6-a055-55e9bd273c6e-kube-api-access-76rr5\") pod \"0bcb6b45-03e0-42f6-a055-55e9bd273c6e\" (UID: \"0bcb6b45-03e0-42f6-a055-55e9bd273c6e\") "
Jul 28 12:41:24.439582 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:24.439553    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bcb6b45-03e0-42f6-a055-55e9bd273c6e-kube-api-access-76rr5" (OuterVolumeSpecName: "kube-api-access-76rr5") pod "0bcb6b45-03e0-42f6-a055-55e9bd273c6e" (UID: "0bcb6b45-03e0-42f6-a055-55e9bd273c6e"). InnerVolumeSpecName "kube-api-access-76rr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:41:24.538538 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:24.538508    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76rr5\" (UniqueName: \"kubernetes.io/projected/0bcb6b45-03e0-42f6-a055-55e9bd273c6e-kube-api-access-76rr5\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:41:25.096530 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:25.096502    2549 generic.go:356] "Generic (PLEG): container finished" podID="0bcb6b45-03e0-42f6-a055-55e9bd273c6e" containerID="4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218" exitCode=0
Jul 28 12:41:25.096925 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:25.096544    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt"
Jul 28 12:41:25.096925 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:25.096600    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt" event={"ID":"0bcb6b45-03e0-42f6-a055-55e9bd273c6e","Type":"ContainerDied","Data":"4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218"}
Jul 28 12:41:25.096925 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:25.096665    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2jzpt" event={"ID":"0bcb6b45-03e0-42f6-a055-55e9bd273c6e","Type":"ContainerDied","Data":"d359f7ab1c8e0d51bbcbb330613268ad7a32e3486a096a536a181229f8084aa9"}
Jul 28 12:41:25.096925 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:25.096689    2549 scope.go:117] "RemoveContainer" containerID="4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218"
Jul 28 12:41:25.105565 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:25.105549    2549 scope.go:117] "RemoveContainer" containerID="4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218"
Jul 28 12:41:25.105823 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:41:25.105807    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218\": container with ID starting with 4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218 not found: ID does not exist" containerID="4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218"
Jul 28 12:41:25.105875 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:25.105830    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218"} err="failed to get container status \"4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218\": rpc error: code = NotFound desc = could not find container \"4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218\": container with ID starting with 4b6a7c5db4cb7a2f81a5e9b529b3d7991ee3382e7d0e4063ba6950fffa231218 not found: ID does not exist"
Jul 28 12:41:25.112831 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:25.112802    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2jzpt"]
Jul 28 12:41:25.118682 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:25.118660    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2jzpt"]
Jul 28 12:41:26.691602 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:26.691564    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bcb6b45-03e0-42f6-a055-55e9bd273c6e" path="/var/lib/kubelet/pods/0bcb6b45-03e0-42f6-a055-55e9bd273c6e/volumes"
Jul 28 12:41:31.508800 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:31.508760    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-kp4jv"
Jul 28 12:41:31.509257 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:31.508824    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-kp4jv"
Jul 28 12:41:31.530172 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:31.530133    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-kp4jv"
Jul 28 12:41:32.150429 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:32.150401    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-kp4jv"
Jul 28 12:41:52.444260 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.444222    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"]
Jul 28 12:41:52.444747 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.444581    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcb6b45-03e0-42f6-a055-55e9bd273c6e" containerName="registry-server"
Jul 28 12:41:52.444747 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.444593    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcb6b45-03e0-42f6-a055-55e9bd273c6e" containerName="registry-server"
Jul 28 12:41:52.444747 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.444690    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcb6b45-03e0-42f6-a055-55e9bd273c6e" containerName="registry-server"
Jul 28 12:41:52.451699 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.451681    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:41:52.454808 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.454784    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mwqd4\""
Jul 28 12:41:52.460027 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.460003    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"]
Jul 28 12:41:52.479089 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.479062    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwqxf\" (UniqueName: \"kubernetes.io/projected/85dee925-0bf3-4f87-b7ab-13bf75c6b15b-kube-api-access-bwqxf\") pod \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" (UID: \"85dee925-0bf3-4f87-b7ab-13bf75c6b15b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:41:52.580577 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.580540    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwqxf\" (UniqueName: \"kubernetes.io/projected/85dee925-0bf3-4f87-b7ab-13bf75c6b15b-kube-api-access-bwqxf\") pod \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" (UID: \"85dee925-0bf3-4f87-b7ab-13bf75c6b15b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:41:52.592240 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.592209    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwqxf\" (UniqueName: \"kubernetes.io/projected/85dee925-0bf3-4f87-b7ab-13bf75c6b15b-kube-api-access-bwqxf\") pod \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" (UID: \"85dee925-0bf3-4f87-b7ab-13bf75c6b15b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:41:52.762117 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.762019    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:41:52.892541 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:52.892355    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"]
Jul 28 12:41:52.895336 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:41:52.895311    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85dee925_0bf3_4f87_b7ab_13bf75c6b15b.slice/crio-9b4b0c8b8c135f93b0057dde2e45f1f3df3c3cb7c2e81eb70f272725eace93ef WatchSource:0}: Error finding container 9b4b0c8b8c135f93b0057dde2e45f1f3df3c3cb7c2e81eb70f272725eace93ef: Status 404 returned error can't find the container with id 9b4b0c8b8c135f93b0057dde2e45f1f3df3c3cb7c2e81eb70f272725eace93ef
Jul 28 12:41:53.226911 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:53.226873    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8" event={"ID":"85dee925-0bf3-4f87-b7ab-13bf75c6b15b","Type":"ContainerStarted","Data":"9b4b0c8b8c135f93b0057dde2e45f1f3df3c3cb7c2e81eb70f272725eace93ef"}
Jul 28 12:41:55.237662 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:55.237551    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8" event={"ID":"85dee925-0bf3-4f87-b7ab-13bf75c6b15b","Type":"ContainerStarted","Data":"3398c855beb99ec8793825df4432a71f599f5d5c2a7b5b4d154c2417eb94d7ad"}
Jul 28 12:41:55.238032 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:55.237682    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:41:55.255833 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:55.255779    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8" podStartSLOduration=1.285359285 podStartE2EDuration="3.25576553s" totalImagesPullingTime="1.970406245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:41:55.254355494 +0000 UTC m=+707.118605354" watchObservedRunningTime="2026-07-28 12:41:55.25576553 +0000 UTC m=+707.120015443"
Jul 28 12:41:56.505535 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:56.505493    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"]
Jul 28 12:41:56.509249 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:56.509225    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"
Jul 28 12:41:56.512059 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:56.512039    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-85krs\""
Jul 28 12:41:56.519396 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:56.519364    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"]
Jul 28 12:41:56.619867 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:56.619835    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5dnh\" (UniqueName: \"kubernetes.io/projected/0d3b7849-db09-41a2-8311-32cad5df1376-kube-api-access-s5dnh\") pod \"authorino-operator-75cc8d5cb7-dmrxr\" (UID: \"0d3b7849-db09-41a2-8311-32cad5df1376\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"
Jul 28 12:41:56.720328 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:56.720297    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5dnh\" (UniqueName: \"kubernetes.io/projected/0d3b7849-db09-41a2-8311-32cad5df1376-kube-api-access-s5dnh\") pod \"authorino-operator-75cc8d5cb7-dmrxr\" (UID: \"0d3b7849-db09-41a2-8311-32cad5df1376\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"
Jul 28 12:41:56.734120 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:56.734089    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5dnh\" (UniqueName: \"kubernetes.io/projected/0d3b7849-db09-41a2-8311-32cad5df1376-kube-api-access-s5dnh\") pod \"authorino-operator-75cc8d5cb7-dmrxr\" (UID: \"0d3b7849-db09-41a2-8311-32cad5df1376\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"
Jul 28 12:41:56.819333 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:56.819303    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"
Jul 28 12:41:56.959972 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:56.959919    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"]
Jul 28 12:41:56.962201 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:41:56.962170    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3b7849_db09_41a2_8311_32cad5df1376.slice/crio-a5eb80596a4b6a4e58033ec6a76592bee736f4193cdc19d8ad5372e5bc04a37d WatchSource:0}: Error finding container a5eb80596a4b6a4e58033ec6a76592bee736f4193cdc19d8ad5372e5bc04a37d: Status 404 returned error can't find the container with id a5eb80596a4b6a4e58033ec6a76592bee736f4193cdc19d8ad5372e5bc04a37d
Jul 28 12:41:57.248297 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:57.248207    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr" event={"ID":"0d3b7849-db09-41a2-8311-32cad5df1376","Type":"ContainerStarted","Data":"a5eb80596a4b6a4e58033ec6a76592bee736f4193cdc19d8ad5372e5bc04a37d"}
Jul 28 12:41:59.261477 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:59.261361    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr" event={"ID":"0d3b7849-db09-41a2-8311-32cad5df1376","Type":"ContainerStarted","Data":"b2e55e2081e83a0a85ba8577b375921eb386df12ad5c89279cb416032d204c12"}
Jul 28 12:41:59.261477 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:59.261477    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"
Jul 28 12:41:59.281315 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:41:59.281253    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr" podStartSLOduration=1.187109601 podStartE2EDuration="3.281218583s" totalImagesPullingTime="2.094108982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:41:59.277823029 +0000 UTC m=+711.142072889" watchObservedRunningTime="2026-07-28 12:41:59.281218583 +0000 UTC m=+711.145468443"
Jul 28 12:42:04.317645 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.316894    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"]
Jul 28 12:42:04.321648 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.321613    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:04.324490 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.324469    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5k6kz\""
Jul 28 12:42:04.331417 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.331392    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"]
Jul 28 12:42:04.385304 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.385276    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tg6b\" (UniqueName: \"kubernetes.io/projected/7de35cd7-ceca-49e0-9ffc-565f9a885360-kube-api-access-6tg6b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" (UID: \"7de35cd7-ceca-49e0-9ffc-565f9a885360\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:04.385462 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.385314    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de35cd7-ceca-49e0-9ffc-565f9a885360-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" (UID: \"7de35cd7-ceca-49e0-9ffc-565f9a885360\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:04.486778 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.486740    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tg6b\" (UniqueName: \"kubernetes.io/projected/7de35cd7-ceca-49e0-9ffc-565f9a885360-kube-api-access-6tg6b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" (UID: \"7de35cd7-ceca-49e0-9ffc-565f9a885360\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:04.486778 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.486782    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de35cd7-ceca-49e0-9ffc-565f9a885360-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" (UID: \"7de35cd7-ceca-49e0-9ffc-565f9a885360\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:04.487163 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.487140    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de35cd7-ceca-49e0-9ffc-565f9a885360-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" (UID: \"7de35cd7-ceca-49e0-9ffc-565f9a885360\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:04.496766 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.496750    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tg6b\" (UniqueName: \"kubernetes.io/projected/7de35cd7-ceca-49e0-9ffc-565f9a885360-kube-api-access-6tg6b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" (UID: \"7de35cd7-ceca-49e0-9ffc-565f9a885360\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:04.632268 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.632176    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:04.765312 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:04.765282    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"]
Jul 28 12:42:04.768375 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:42:04.768340    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de35cd7_ceca_49e0_9ffc_565f9a885360.slice/crio-1cc5b8cfa48393c010d2b3bfde4f7f3dbe6bf6a0c4522ce79bb9e8693e419eb4 WatchSource:0}: Error finding container 1cc5b8cfa48393c010d2b3bfde4f7f3dbe6bf6a0c4522ce79bb9e8693e419eb4: Status 404 returned error can't find the container with id 1cc5b8cfa48393c010d2b3bfde4f7f3dbe6bf6a0c4522ce79bb9e8693e419eb4
Jul 28 12:42:05.292800 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:05.292766    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg" event={"ID":"7de35cd7-ceca-49e0-9ffc-565f9a885360","Type":"ContainerStarted","Data":"1cc5b8cfa48393c010d2b3bfde4f7f3dbe6bf6a0c4522ce79bb9e8693e419eb4"}
Jul 28 12:42:06.246008 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:06.245971    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:10.268041 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:10.268003    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dmrxr"
Jul 28 12:42:10.320086 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:10.320051    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg" event={"ID":"7de35cd7-ceca-49e0-9ffc-565f9a885360","Type":"ContainerStarted","Data":"cbabc154b97f8c7c0b1ee079c3eead77b2ffcb17fdee950481ae138213e69e4a"}
Jul 28 12:42:10.320245 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:10.320114    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:10.343436 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:10.343386    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg" podStartSLOduration=1.804937036 podStartE2EDuration="6.343368182s" totalImagesPullingTime="4.538431146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:42:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:42:10.341834446 +0000 UTC m=+722.206084310" watchObservedRunningTime="2026-07-28 12:42:10.343368182 +0000 UTC m=+722.207618042"
Jul 28 12:42:21.326805 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:21.326774    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:23.123405 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.123371    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"]
Jul 28 12:42:23.123851 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.123549    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" containerName="manager" containerID="cri-o://cbabc154b97f8c7c0b1ee079c3eead77b2ffcb17fdee950481ae138213e69e4a" gracePeriod=2
Jul 28 12:42:23.137805 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.137777    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"]
Jul 28 12:42:23.140821 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.140790    2549 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7de35cd7-ceca-49e0-9ffc-565f9a885360\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\": pods \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:23.156544 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.156518    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"]
Jul 28 12:42:23.156821 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.156767    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" containerName="manager" containerID="cri-o://3398c855beb99ec8793825df4432a71f599f5d5c2a7b5b4d154c2417eb94d7ad" gracePeriod=2
Jul 28 12:42:23.164247 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.164218    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"]
Jul 28 12:42:23.164783 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.164753    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" containerName="manager"
Jul 28 12:42:23.164783 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.164775    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" containerName="manager"
Jul 28 12:42:23.164942 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.164888    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" containerName="manager"
Jul 28 12:42:23.166252 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.166223    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:23.167978 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.167964    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"]
Jul 28 12:42:23.168054 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.168044    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:23.173534 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.173508    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:23.177763 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.177739    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:23.184581 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.184557    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"]
Jul 28 12:42:23.191390 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.191203    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"]
Jul 28 12:42:23.191724 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.191703    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" containerName="manager"
Jul 28 12:42:23.191724 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.191725    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" containerName="manager"
Jul 28 12:42:23.191911 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.191843    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" containerName="manager"
Jul 28 12:42:23.195730 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.195710    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"
Jul 28 12:42:23.208530 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.208509    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"]
Jul 28 12:42:23.229460 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.229419    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:23.231980 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.231944    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:23.256990 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.256967    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kts4m\" (UID: \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:23.257099 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.256996    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjtft\" (UniqueName: \"kubernetes.io/projected/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-kube-api-access-sjtft\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kts4m\" (UID: \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:23.358285 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.358250    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kts4m\" (UID: \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:23.358285 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.358290    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjtft\" (UniqueName: \"kubernetes.io/projected/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-kube-api-access-sjtft\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kts4m\" (UID: \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:23.358482 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.358316    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-988k8\" (UniqueName: \"kubernetes.io/projected/4f65d3d2-81e7-4ea9-98bd-e0fa933637c0-kube-api-access-988k8\") pod \"limitador-operator-controller-manager-859d7b78bd-2c98v\" (UID: \"4f65d3d2-81e7-4ea9-98bd-e0fa933637c0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"
Jul 28 12:42:23.358806 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.358777    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kts4m\" (UID: \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:23.370462 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.370439    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjtft\" (UniqueName: \"kubernetes.io/projected/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-kube-api-access-sjtft\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kts4m\" (UID: \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:23.382927 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.382901    2549 generic.go:356] "Generic (PLEG): container finished" podID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" containerID="3398c855beb99ec8793825df4432a71f599f5d5c2a7b5b4d154c2417eb94d7ad" exitCode=0
Jul 28 12:42:23.384456 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.384433    2549 generic.go:356] "Generic (PLEG): container finished" podID="7de35cd7-ceca-49e0-9ffc-565f9a885360" containerID="cbabc154b97f8c7c0b1ee079c3eead77b2ffcb17fdee950481ae138213e69e4a" exitCode=0
Jul 28 12:42:23.399401 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.399382    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:23.402161 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.402143    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:23.402254 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.402222    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:23.404741 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.404717    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:23.407037 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.407016    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:23.409478 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:23.409453    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:23.459224 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.459199    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-988k8\" (UniqueName: \"kubernetes.io/projected/4f65d3d2-81e7-4ea9-98bd-e0fa933637c0-kube-api-access-988k8\") pod \"limitador-operator-controller-manager-859d7b78bd-2c98v\" (UID: \"4f65d3d2-81e7-4ea9-98bd-e0fa933637c0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"
Jul 28 12:42:23.473322 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.473302    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-988k8\" (UniqueName: \"kubernetes.io/projected/4f65d3d2-81e7-4ea9-98bd-e0fa933637c0-kube-api-access-988k8\") pod \"limitador-operator-controller-manager-859d7b78bd-2c98v\" (UID: \"4f65d3d2-81e7-4ea9-98bd-e0fa933637c0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"
Jul 28 12:42:23.559042 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.559008    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:23.560211 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.560190    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwqxf\" (UniqueName: \"kubernetes.io/projected/85dee925-0bf3-4f87-b7ab-13bf75c6b15b-kube-api-access-bwqxf\") pod \"85dee925-0bf3-4f87-b7ab-13bf75c6b15b\" (UID: \"85dee925-0bf3-4f87-b7ab-13bf75c6b15b\") "
Jul 28 12:42:23.560314 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.560276    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tg6b\" (UniqueName: \"kubernetes.io/projected/7de35cd7-ceca-49e0-9ffc-565f9a885360-kube-api-access-6tg6b\") pod \"7de35cd7-ceca-49e0-9ffc-565f9a885360\" (UID: \"7de35cd7-ceca-49e0-9ffc-565f9a885360\") "
Jul 28 12:42:23.560393 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.560373    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de35cd7-ceca-49e0-9ffc-565f9a885360-extensions-socket-volume\") pod \"7de35cd7-ceca-49e0-9ffc-565f9a885360\" (UID: \"7de35cd7-ceca-49e0-9ffc-565f9a885360\") "
Jul 28 12:42:23.560934 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.560860    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de35cd7-ceca-49e0-9ffc-565f9a885360-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7de35cd7-ceca-49e0-9ffc-565f9a885360" (UID: "7de35cd7-ceca-49e0-9ffc-565f9a885360"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:42:23.562425 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.562398    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de35cd7-ceca-49e0-9ffc-565f9a885360-kube-api-access-6tg6b" (OuterVolumeSpecName: "kube-api-access-6tg6b") pod "7de35cd7-ceca-49e0-9ffc-565f9a885360" (UID: "7de35cd7-ceca-49e0-9ffc-565f9a885360"). InnerVolumeSpecName "kube-api-access-6tg6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:42:23.562598 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.562582    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85dee925-0bf3-4f87-b7ab-13bf75c6b15b-kube-api-access-bwqxf" (OuterVolumeSpecName: "kube-api-access-bwqxf") pod "85dee925-0bf3-4f87-b7ab-13bf75c6b15b" (UID: "85dee925-0bf3-4f87-b7ab-13bf75c6b15b"). InnerVolumeSpecName "kube-api-access-bwqxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:42:23.564745 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.564730    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"
Jul 28 12:42:23.661474 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.661397    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwqxf\" (UniqueName: \"kubernetes.io/projected/85dee925-0bf3-4f87-b7ab-13bf75c6b15b-kube-api-access-bwqxf\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:42:23.661474 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.661426    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tg6b\" (UniqueName: \"kubernetes.io/projected/7de35cd7-ceca-49e0-9ffc-565f9a885360-kube-api-access-6tg6b\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:42:23.661474 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.661437    2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de35cd7-ceca-49e0-9ffc-565f9a885360-extensions-socket-volume\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:42:23.724714 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.724587    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"]
Jul 28 12:42:23.727306 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:42:23.727282    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e3498a3_08c8_419e_b5ec_5d0dc4ef031a.slice/crio-8b07e7ae26ecd8917133145ec49f9d34fa52a2e4e15d178fbff9b9279244341c WatchSource:0}: Error finding container 8b07e7ae26ecd8917133145ec49f9d34fa52a2e4e15d178fbff9b9279244341c: Status 404 returned error can't find the container with id 8b07e7ae26ecd8917133145ec49f9d34fa52a2e4e15d178fbff9b9279244341c
Jul 28 12:42:23.739753 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:23.739734    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"]
Jul 28 12:42:23.746133 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:42:23.746108    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f65d3d2_81e7_4ea9_98bd_e0fa933637c0.slice/crio-9c234924a203fbae2f36c62c9ee552d759ab545bde18ea2bfd960432c59ab169 WatchSource:0}: Error finding container 9c234924a203fbae2f36c62c9ee552d759ab545bde18ea2bfd960432c59ab169: Status 404 returned error can't find the container with id 9c234924a203fbae2f36c62c9ee552d759ab545bde18ea2bfd960432c59ab169
Jul 28 12:42:24.390990 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.390959    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:24.391440 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.390964    2549 scope.go:117] "RemoveContainer" containerID="3398c855beb99ec8793825df4432a71f599f5d5c2a7b5b4d154c2417eb94d7ad"
Jul 28 12:42:24.392489 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.392466    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:24.394075 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:24.394041    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:24.395839 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.395777    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v" event={"ID":"4f65d3d2-81e7-4ea9-98bd-e0fa933637c0","Type":"ContainerStarted","Data":"2ffae43a9a490cb7f3d4547d072039d077d43722fb161199838ed333353ed301"}
Jul 28 12:42:24.395950 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.395875    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v" event={"ID":"4f65d3d2-81e7-4ea9-98bd-e0fa933637c0","Type":"ContainerStarted","Data":"9c234924a203fbae2f36c62c9ee552d759ab545bde18ea2bfd960432c59ab169"}
Jul 28 12:42:24.397117 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:24.396455    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:24.397117 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.396959    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"
Jul 28 12:42:24.400110 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:24.400081    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:24.404127 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.404098    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m" event={"ID":"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a","Type":"ContainerStarted","Data":"79390eb84b72fba021505caa957ddbf5131ea5a3f79e544ca2d0f7938db06b5f"}
Jul 28 12:42:24.404232 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.404131    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m" event={"ID":"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a","Type":"ContainerStarted","Data":"8b07e7ae26ecd8917133145ec49f9d34fa52a2e4e15d178fbff9b9279244341c"}
Jul 28 12:42:24.404298 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.404244    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:24.407289 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.407273    2549 scope.go:117] "RemoveContainer" containerID="cbabc154b97f8c7c0b1ee079c3eead77b2ffcb17fdee950481ae138213e69e4a"
Jul 28 12:42:24.421214 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.421174    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v" podStartSLOduration=1.421161835 podStartE2EDuration="1.421161835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:42:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:42:24.419711996 +0000 UTC m=+736.283961855" watchObservedRunningTime="2026-07-28 12:42:24.421161835 +0000 UTC m=+736.285411694"
Jul 28 12:42:24.422152 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:24.422131    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:24.424706 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:24.424683    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4p5l8\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4p5l8"
Jul 28 12:42:24.426969 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:42:24.426949    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qsvbg\" is forbidden: User \"system:node:ip-10-0-132-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-90.ec2.internal' and this object" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qsvbg"
Jul 28 12:42:24.452658 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.452596    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m" podStartSLOduration=1.452583013 podStartE2EDuration="1.452583013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:42:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:42:24.450734753 +0000 UTC m=+736.314984613" watchObservedRunningTime="2026-07-28 12:42:24.452583013 +0000 UTC m=+736.316832953"
Jul 28 12:42:24.692241 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.692163    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7de35cd7-ceca-49e0-9ffc-565f9a885360" path="/var/lib/kubelet/pods/7de35cd7-ceca-49e0-9ffc-565f9a885360/volumes"
Jul 28 12:42:24.692483 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:24.692470    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85dee925-0bf3-4f87-b7ab-13bf75c6b15b" path="/var/lib/kubelet/pods/85dee925-0bf3-4f87-b7ab-13bf75c6b15b/volumes"
Jul 28 12:42:35.413351 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:35.413315    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:36.418222 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:36.418185    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2c98v"
Jul 28 12:42:39.334867 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.334836    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"]
Jul 28 12:42:39.335257 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.335025    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m" podUID="9e3498a3-08c8-419e-b5ec-5d0dc4ef031a" containerName="manager" containerID="cri-o://79390eb84b72fba021505caa957ddbf5131ea5a3f79e544ca2d0f7938db06b5f" gracePeriod=10
Jul 28 12:42:39.479606 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.479574    2549 generic.go:356] "Generic (PLEG): container finished" podID="9e3498a3-08c8-419e-b5ec-5d0dc4ef031a" containerID="79390eb84b72fba021505caa957ddbf5131ea5a3f79e544ca2d0f7938db06b5f" exitCode=0
Jul 28 12:42:39.479782 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.479661    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m" event={"ID":"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a","Type":"ContainerDied","Data":"79390eb84b72fba021505caa957ddbf5131ea5a3f79e544ca2d0f7938db06b5f"}
Jul 28 12:42:39.589227 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.589165    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:39.698885 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.698854    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-extensions-socket-volume\") pod \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\" (UID: \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\") "
Jul 28 12:42:39.699059 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.698983    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjtft\" (UniqueName: \"kubernetes.io/projected/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-kube-api-access-sjtft\") pod \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\" (UID: \"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a\") "
Jul 28 12:42:39.699234 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.699210    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9e3498a3-08c8-419e-b5ec-5d0dc4ef031a" (UID: "9e3498a3-08c8-419e-b5ec-5d0dc4ef031a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 12:42:39.701072 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.701053    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-kube-api-access-sjtft" (OuterVolumeSpecName: "kube-api-access-sjtft") pod "9e3498a3-08c8-419e-b5ec-5d0dc4ef031a" (UID: "9e3498a3-08c8-419e-b5ec-5d0dc4ef031a"). InnerVolumeSpecName "kube-api-access-sjtft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:42:39.799780 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.799745    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjtft\" (UniqueName: \"kubernetes.io/projected/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-kube-api-access-sjtft\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:42:39.799780 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:39.799773    2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a-extensions-socket-volume\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:42:40.485827 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:40.485793    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m" event={"ID":"9e3498a3-08c8-419e-b5ec-5d0dc4ef031a","Type":"ContainerDied","Data":"8b07e7ae26ecd8917133145ec49f9d34fa52a2e4e15d178fbff9b9279244341c"}
Jul 28 12:42:40.485827 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:40.485811    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"
Jul 28 12:42:40.486300 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:40.485880    2549 scope.go:117] "RemoveContainer" containerID="79390eb84b72fba021505caa957ddbf5131ea5a3f79e544ca2d0f7938db06b5f"
Jul 28 12:42:40.512340 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:40.512312    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"]
Jul 28 12:42:40.516090 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:40.516066    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kts4m"]
Jul 28 12:42:40.692052 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:40.692016    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e3498a3-08c8-419e-b5ec-5d0dc4ef031a" path="/var/lib/kubelet/pods/9e3498a3-08c8-419e-b5ec-5d0dc4ef031a/volumes"
Jul 28 12:42:55.828458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.828416    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"]
Jul 28 12:42:55.828966 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.828925    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e3498a3-08c8-419e-b5ec-5d0dc4ef031a" containerName="manager"
Jul 28 12:42:55.828966 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.828945    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e3498a3-08c8-419e-b5ec-5d0dc4ef031a" containerName="manager"
Jul 28 12:42:55.829079 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.829064    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e3498a3-08c8-419e-b5ec-5d0dc4ef031a" containerName="manager"
Jul 28 12:42:55.832330 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.832311    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:55.835095 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.835078    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-thh6q\""
Jul 28 12:42:55.844192 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.844162    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"]
Jul 28 12:42:55.937192 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.937152    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:55.937192 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.937194    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db993d21-5f17-4559-94b3-075a8a6acac2-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:55.937400 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.937278    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:55.937400 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.937301    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98tfp\" (UniqueName: \"kubernetes.io/projected/db993d21-5f17-4559-94b3-075a8a6acac2-kube-api-access-98tfp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:55.937400 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.937341    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:55.937400 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.937358    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:55.937400 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.937387    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db993d21-5f17-4559-94b3-075a8a6acac2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:55.937554 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.937424    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:55.937554 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:55.937451    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db993d21-5f17-4559-94b3-075a8a6acac2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.038458 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038417    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.038651 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038466    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db993d21-5f17-4559-94b3-075a8a6acac2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.038651 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038503    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.038651 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038528    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db993d21-5f17-4559-94b3-075a8a6acac2-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.038651 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038561    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.038868 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038672    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98tfp\" (UniqueName: \"kubernetes.io/projected/db993d21-5f17-4559-94b3-075a8a6acac2-kube-api-access-98tfp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.038868 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038742    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.038868 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038772    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.038868 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038804    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db993d21-5f17-4559-94b3-075a8a6acac2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.039080 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.038923    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.039080 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.039050    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.039175 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.039153    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.039286 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.039265    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db993d21-5f17-4559-94b3-075a8a6acac2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.039328 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.039271    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.040885 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.040856    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db993d21-5f17-4559-94b3-075a8a6acac2-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.041322 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.041305    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db993d21-5f17-4559-94b3-075a8a6acac2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.047713 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.047619    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db993d21-5f17-4559-94b3-075a8a6acac2-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.048534 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.048502    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98tfp\" (UniqueName: \"kubernetes.io/projected/db993d21-5f17-4559-94b3-075a8a6acac2-kube-api-access-98tfp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j4qxs\" (UID: \"db993d21-5f17-4559-94b3-075a8a6acac2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.143477 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.143381    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:56.281135 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.281110    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"]
Jul 28 12:42:56.283350 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:42:56.283326    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb993d21_5f17_4559_94b3_075a8a6acac2.slice/crio-f6a31064e7d97e6592e300e409044f995656d5046ce2550d7fd1ef506dcb045b WatchSource:0}: Error finding container f6a31064e7d97e6592e300e409044f995656d5046ce2550d7fd1ef506dcb045b: Status 404 returned error can't find the container with id f6a31064e7d97e6592e300e409044f995656d5046ce2550d7fd1ef506dcb045b
Jul 28 12:42:56.285847 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.285812    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:42:56.285946 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.285893    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:42:56.285946 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.285923    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:42:56.569512 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.569479    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" event={"ID":"db993d21-5f17-4559-94b3-075a8a6acac2","Type":"ContainerStarted","Data":"d1b6faf3ac7c37827de4a8a6513c6ccedd6984446dbd9598055ee476b79bdafd"}
Jul 28 12:42:56.569703 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.569534    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" event={"ID":"db993d21-5f17-4559-94b3-075a8a6acac2","Type":"ContainerStarted","Data":"f6a31064e7d97e6592e300e409044f995656d5046ce2550d7fd1ef506dcb045b"}
Jul 28 12:42:56.594293 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:56.594238    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" podStartSLOduration=1.594224911 podStartE2EDuration="1.594224911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:42:56.592576122 +0000 UTC m=+768.456825981" watchObservedRunningTime="2026-07-28 12:42:56.594224911 +0000 UTC m=+768.458474769"
Jul 28 12:42:57.144410 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:57.144367    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:57.149238 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:57.149214    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:57.576202 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:57.576163    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:42:57.577137 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:42:57.577115    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:43:14.042734 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.042697    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rsggc"]
Jul 28 12:43:14.047695 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.047674    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:14.050787 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.050765    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tffwh\""
Jul 28 12:43:14.050897 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.050783    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 12:43:14.056093 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.056061    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rsggc"]
Jul 28 12:43:14.136710 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.136677    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rsggc"]
Jul 28 12:43:14.196791 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.196759    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/98e4181b-fc8f-49a9-9751-929ad9924da7-kube-api-access-4lt5n\") pod \"limitador-limitador-64c94fd64f-rsggc\" (UID: \"98e4181b-fc8f-49a9-9751-929ad9924da7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:14.196936 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.196800    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98e4181b-fc8f-49a9-9751-929ad9924da7-config-file\") pod \"limitador-limitador-64c94fd64f-rsggc\" (UID: \"98e4181b-fc8f-49a9-9751-929ad9924da7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:14.298103 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.298023    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98e4181b-fc8f-49a9-9751-929ad9924da7-config-file\") pod \"limitador-limitador-64c94fd64f-rsggc\" (UID: \"98e4181b-fc8f-49a9-9751-929ad9924da7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:14.298259 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.298132    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/98e4181b-fc8f-49a9-9751-929ad9924da7-kube-api-access-4lt5n\") pod \"limitador-limitador-64c94fd64f-rsggc\" (UID: \"98e4181b-fc8f-49a9-9751-929ad9924da7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:14.298691 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.298673    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98e4181b-fc8f-49a9-9751-929ad9924da7-config-file\") pod \"limitador-limitador-64c94fd64f-rsggc\" (UID: \"98e4181b-fc8f-49a9-9751-929ad9924da7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:14.308697 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.308674    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/98e4181b-fc8f-49a9-9751-929ad9924da7-kube-api-access-4lt5n\") pod \"limitador-limitador-64c94fd64f-rsggc\" (UID: \"98e4181b-fc8f-49a9-9751-929ad9924da7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:14.358708 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.358672    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:14.502166 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.502090    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rsggc"]
Jul 28 12:43:14.504584 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:43:14.504553    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e4181b_fc8f_49a9_9751_929ad9924da7.slice/crio-369f6d4975b11381652dc9d7e10c63663abb3375292acd8c05b7daa06d73ed07 WatchSource:0}: Error finding container 369f6d4975b11381652dc9d7e10c63663abb3375292acd8c05b7daa06d73ed07: Status 404 returned error can't find the container with id 369f6d4975b11381652dc9d7e10c63663abb3375292acd8c05b7daa06d73ed07
Jul 28 12:43:14.652470 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.652432    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc" event={"ID":"98e4181b-fc8f-49a9-9751-929ad9924da7","Type":"ContainerStarted","Data":"369f6d4975b11381652dc9d7e10c63663abb3375292acd8c05b7daa06d73ed07"}
Jul 28 12:43:14.824643 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.824600    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-5whhl"]
Jul 28 12:43:14.829101 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.829074    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5whhl"
Jul 28 12:43:14.832149 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.832127    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wf4f5\""
Jul 28 12:43:14.835324 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:14.835299    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5whhl"]
Jul 28 12:43:15.004485 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:15.004398    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7fql\" (UniqueName: \"kubernetes.io/projected/208e06a3-9a40-4076-9820-09715901e9d5-kube-api-access-l7fql\") pod \"authorino-58cd955675-5whhl\" (UID: \"208e06a3-9a40-4076-9820-09715901e9d5\") " pod="kuadrant-system/authorino-58cd955675-5whhl"
Jul 28 12:43:15.105592 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:15.105561    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7fql\" (UniqueName: \"kubernetes.io/projected/208e06a3-9a40-4076-9820-09715901e9d5-kube-api-access-l7fql\") pod \"authorino-58cd955675-5whhl\" (UID: \"208e06a3-9a40-4076-9820-09715901e9d5\") " pod="kuadrant-system/authorino-58cd955675-5whhl"
Jul 28 12:43:15.115022 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:15.114998    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7fql\" (UniqueName: \"kubernetes.io/projected/208e06a3-9a40-4076-9820-09715901e9d5-kube-api-access-l7fql\") pod \"authorino-58cd955675-5whhl\" (UID: \"208e06a3-9a40-4076-9820-09715901e9d5\") " pod="kuadrant-system/authorino-58cd955675-5whhl"
Jul 28 12:43:15.139855 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:15.139829    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5whhl"
Jul 28 12:43:15.286710 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:15.286579    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5whhl"]
Jul 28 12:43:15.290509 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:43:15.290476    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod208e06a3_9a40_4076_9820_09715901e9d5.slice/crio-b0b0cea873280ca30872208426b1c37366de382e8097025924f36244f4f7afe6 WatchSource:0}: Error finding container b0b0cea873280ca30872208426b1c37366de382e8097025924f36244f4f7afe6: Status 404 returned error can't find the container with id b0b0cea873280ca30872208426b1c37366de382e8097025924f36244f4f7afe6
Jul 28 12:43:15.659286 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:15.659249    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5whhl" event={"ID":"208e06a3-9a40-4076-9820-09715901e9d5","Type":"ContainerStarted","Data":"b0b0cea873280ca30872208426b1c37366de382e8097025924f36244f4f7afe6"}
Jul 28 12:43:19.105484 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:19.105448    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5whhl"]
Jul 28 12:43:19.680960 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:19.680924    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc" event={"ID":"98e4181b-fc8f-49a9-9751-929ad9924da7","Type":"ContainerStarted","Data":"b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47"}
Jul 28 12:43:19.681165 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:19.680999    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:19.682323 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:19.682302    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5whhl" event={"ID":"208e06a3-9a40-4076-9820-09715901e9d5","Type":"ContainerStarted","Data":"e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232"}
Jul 28 12:43:19.682447 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:19.682379    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-5whhl" podUID="208e06a3-9a40-4076-9820-09715901e9d5" containerName="authorino" containerID="cri-o://e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232" gracePeriod=30
Jul 28 12:43:19.697992 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:19.697947    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc" podStartSLOduration=1.085994492 podStartE2EDuration="5.697932404s" totalImagesPullingTime="4.611937912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:43:19.696010136 +0000 UTC m=+791.560260021" watchObservedRunningTime="2026-07-28 12:43:19.697932404 +0000 UTC m=+791.562182263"
Jul 28 12:43:19.713038 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:19.712978    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-5whhl" podStartSLOduration=1.841075603 podStartE2EDuration="5.71295986s" totalImagesPullingTime="3.871884257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:43:19.710459301 +0000 UTC m=+791.574709161" watchObservedRunningTime="2026-07-28 12:43:19.71295986 +0000 UTC m=+791.577209728"
Jul 28 12:43:19.930092 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:19.930071    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5whhl"
Jul 28 12:43:20.050357 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.050277    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7fql\" (UniqueName: \"kubernetes.io/projected/208e06a3-9a40-4076-9820-09715901e9d5-kube-api-access-l7fql\") pod \"208e06a3-9a40-4076-9820-09715901e9d5\" (UID: \"208e06a3-9a40-4076-9820-09715901e9d5\") "
Jul 28 12:43:20.052429 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.052407    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/208e06a3-9a40-4076-9820-09715901e9d5-kube-api-access-l7fql" (OuterVolumeSpecName: "kube-api-access-l7fql") pod "208e06a3-9a40-4076-9820-09715901e9d5" (UID: "208e06a3-9a40-4076-9820-09715901e9d5"). InnerVolumeSpecName "kube-api-access-l7fql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:43:20.151492 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.151455    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7fql\" (UniqueName: \"kubernetes.io/projected/208e06a3-9a40-4076-9820-09715901e9d5-kube-api-access-l7fql\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:43:20.688267 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.688236    2549 generic.go:356] "Generic (PLEG): container finished" podID="208e06a3-9a40-4076-9820-09715901e9d5" containerID="e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232" exitCode=0
Jul 28 12:43:20.688444 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.688337    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5whhl"
Jul 28 12:43:20.692445 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.692402    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5whhl" event={"ID":"208e06a3-9a40-4076-9820-09715901e9d5","Type":"ContainerDied","Data":"e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232"}
Jul 28 12:43:20.692445 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.692446    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5whhl" event={"ID":"208e06a3-9a40-4076-9820-09715901e9d5","Type":"ContainerDied","Data":"b0b0cea873280ca30872208426b1c37366de382e8097025924f36244f4f7afe6"}
Jul 28 12:43:20.692672 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.692471    2549 scope.go:117] "RemoveContainer" containerID="e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232"
Jul 28 12:43:20.701773 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.701755    2549 scope.go:117] "RemoveContainer" containerID="e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232"
Jul 28 12:43:20.702025 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:43:20.702006    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232\": container with ID starting with e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232 not found: ID does not exist" containerID="e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232"
Jul 28 12:43:20.702075 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.702034    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232"} err="failed to get container status \"e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232\": rpc error: code = NotFound desc = could not find container \"e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232\": container with ID starting with e28acfa53a02a7119f17e0e6384c1106eca0fee4e34772d02bff26c4eeef6232 not found: ID does not exist"
Jul 28 12:43:20.713318 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.713298    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5whhl"]
Jul 28 12:43:20.717473 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:20.717442    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-5whhl"]
Jul 28 12:43:22.692186 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:22.692151    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="208e06a3-9a40-4076-9820-09715901e9d5" path="/var/lib/kubelet/pods/208e06a3-9a40-4076-9820-09715901e9d5/volumes"
Jul 28 12:43:28.582688 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:28.582581    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rsggc"]
Jul 28 12:43:28.583234 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:28.582847    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc" podUID="98e4181b-fc8f-49a9-9751-929ad9924da7" containerName="limitador" containerID="cri-o://b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47" gracePeriod=30
Jul 28 12:43:28.583552 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:28.583477    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:29.136467 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.136444    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:29.228742 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.228665    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/98e4181b-fc8f-49a9-9751-929ad9924da7-kube-api-access-4lt5n\") pod \"98e4181b-fc8f-49a9-9751-929ad9924da7\" (UID: \"98e4181b-fc8f-49a9-9751-929ad9924da7\") "
Jul 28 12:43:29.228905 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.228751    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98e4181b-fc8f-49a9-9751-929ad9924da7-config-file\") pod \"98e4181b-fc8f-49a9-9751-929ad9924da7\" (UID: \"98e4181b-fc8f-49a9-9751-929ad9924da7\") "
Jul 28 12:43:29.229138 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.229105    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98e4181b-fc8f-49a9-9751-929ad9924da7-config-file" (OuterVolumeSpecName: "config-file") pod "98e4181b-fc8f-49a9-9751-929ad9924da7" (UID: "98e4181b-fc8f-49a9-9751-929ad9924da7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 12:43:29.230707 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.230686    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98e4181b-fc8f-49a9-9751-929ad9924da7-kube-api-access-4lt5n" (OuterVolumeSpecName: "kube-api-access-4lt5n") pod "98e4181b-fc8f-49a9-9751-929ad9924da7" (UID: "98e4181b-fc8f-49a9-9751-929ad9924da7"). InnerVolumeSpecName "kube-api-access-4lt5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:43:29.329841 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.329814    2549 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98e4181b-fc8f-49a9-9751-929ad9924da7-config-file\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:43:29.329841 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.329837    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/98e4181b-fc8f-49a9-9751-929ad9924da7-kube-api-access-4lt5n\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:43:29.729195 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.729163    2549 generic.go:356] "Generic (PLEG): container finished" podID="98e4181b-fc8f-49a9-9751-929ad9924da7" containerID="b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47" exitCode=0
Jul 28 12:43:29.729597 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.729214    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc"
Jul 28 12:43:29.729597 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.729214    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc" event={"ID":"98e4181b-fc8f-49a9-9751-929ad9924da7","Type":"ContainerDied","Data":"b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47"}
Jul 28 12:43:29.729597 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.729325    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rsggc" event={"ID":"98e4181b-fc8f-49a9-9751-929ad9924da7","Type":"ContainerDied","Data":"369f6d4975b11381652dc9d7e10c63663abb3375292acd8c05b7daa06d73ed07"}
Jul 28 12:43:29.729597 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.729355    2549 scope.go:117] "RemoveContainer" containerID="b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47"
Jul 28 12:43:29.739031 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.739013    2549 scope.go:117] "RemoveContainer" containerID="b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47"
Jul 28 12:43:29.739297 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:43:29.739278    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47\": container with ID starting with b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47 not found: ID does not exist" containerID="b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47"
Jul 28 12:43:29.739361 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.739304    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47"} err="failed to get container status \"b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47\": rpc error: code = NotFound desc = could not find container \"b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47\": container with ID starting with b7c2be2373fa33f54127e3a15d548f77737a3ffd40a1ec07f668c2d14b17af47 not found: ID does not exist"
Jul 28 12:43:29.751616 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.751590    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rsggc"]
Jul 28 12:43:29.755552 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:29.755530    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rsggc"]
Jul 28 12:43:30.692000 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:30.691967    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98e4181b-fc8f-49a9-9751-929ad9924da7" path="/var/lib/kubelet/pods/98e4181b-fc8f-49a9-9751-929ad9924da7/volumes"
Jul 28 12:43:35.062096 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.062059    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"]
Jul 28 12:43:35.062594 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.062573    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98e4181b-fc8f-49a9-9751-929ad9924da7" containerName="limitador"
Jul 28 12:43:35.062706 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.062596    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="98e4181b-fc8f-49a9-9751-929ad9924da7" containerName="limitador"
Jul 28 12:43:35.062706 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.062656    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208e06a3-9a40-4076-9820-09715901e9d5" containerName="authorino"
Jul 28 12:43:35.062706 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.062666    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="208e06a3-9a40-4076-9820-09715901e9d5" containerName="authorino"
Jul 28 12:43:35.062854 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.062757    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="208e06a3-9a40-4076-9820-09715901e9d5" containerName="authorino"
Jul 28 12:43:35.062854 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.062777    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="98e4181b-fc8f-49a9-9751-929ad9924da7" containerName="limitador"
Jul 28 12:43:35.066253 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.066228    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:35.069259 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.069241    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 12:43:35.070747 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.070731    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-dhclm\""
Jul 28 12:43:35.070747 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.070740    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 12:43:35.070906 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.070770    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 28 12:43:35.075422 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.075398    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"]
Jul 28 12:43:35.075782 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.075760    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdnzf\" (UniqueName: \"kubernetes.io/projected/6001921c-2d0b-47db-8c64-90f64c49ed8b-kube-api-access-wdnzf\") pod \"postgres-69fb4594cf-xffsd\" (UID: \"6001921c-2d0b-47db-8c64-90f64c49ed8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:35.075872 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.075817    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6001921c-2d0b-47db-8c64-90f64c49ed8b-data\") pod \"postgres-69fb4594cf-xffsd\" (UID: \"6001921c-2d0b-47db-8c64-90f64c49ed8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:35.176765 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.176728    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdnzf\" (UniqueName: \"kubernetes.io/projected/6001921c-2d0b-47db-8c64-90f64c49ed8b-kube-api-access-wdnzf\") pod \"postgres-69fb4594cf-xffsd\" (UID: \"6001921c-2d0b-47db-8c64-90f64c49ed8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:35.176926 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.176799    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6001921c-2d0b-47db-8c64-90f64c49ed8b-data\") pod \"postgres-69fb4594cf-xffsd\" (UID: \"6001921c-2d0b-47db-8c64-90f64c49ed8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:35.177155 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.177139    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6001921c-2d0b-47db-8c64-90f64c49ed8b-data\") pod \"postgres-69fb4594cf-xffsd\" (UID: \"6001921c-2d0b-47db-8c64-90f64c49ed8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:35.185403 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.185382    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdnzf\" (UniqueName: \"kubernetes.io/projected/6001921c-2d0b-47db-8c64-90f64c49ed8b-kube-api-access-wdnzf\") pod \"postgres-69fb4594cf-xffsd\" (UID: \"6001921c-2d0b-47db-8c64-90f64c49ed8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:35.377841 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.377762    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:35.510438 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.510412    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"]
Jul 28 12:43:35.513345 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:43:35.513322    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6001921c_2d0b_47db_8c64_90f64c49ed8b.slice/crio-43320e57ad21b2c6202aea49560bca5ab986ff6b182b4d469cca90bca32d9f8e WatchSource:0}: Error finding container 43320e57ad21b2c6202aea49560bca5ab986ff6b182b4d469cca90bca32d9f8e: Status 404 returned error can't find the container with id 43320e57ad21b2c6202aea49560bca5ab986ff6b182b4d469cca90bca32d9f8e
Jul 28 12:43:35.757375 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:35.757293    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd" event={"ID":"6001921c-2d0b-47db-8c64-90f64c49ed8b","Type":"ContainerStarted","Data":"43320e57ad21b2c6202aea49560bca5ab986ff6b182b4d469cca90bca32d9f8e"}
Jul 28 12:43:40.785163 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:40.785136    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd" event={"ID":"6001921c-2d0b-47db-8c64-90f64c49ed8b","Type":"ContainerStarted","Data":"1a62a153c6e2ea4418c95eaf8ddda93f352ad28d5df6de04ce67fa219091dc88"}
Jul 28 12:43:40.785539 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:40.785273    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:40.805074 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:40.805033    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd" podStartSLOduration=0.61890293 podStartE2EDuration="5.805019705s" totalImagesPullingTime="5.186116775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:43:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:43:40.803795869 +0000 UTC m=+812.668045727" watchObservedRunningTime="2026-07-28 12:43:40.805019705 +0000 UTC m=+812.669269564"
Jul 28 12:43:46.820050 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:46.820011    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xffsd"
Jul 28 12:43:47.483724 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.483691    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lf2q5"]
Jul 28 12:43:47.487212 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.487196    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lf2q5"
Jul 28 12:43:47.489969 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.489944    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wf4f5\""
Jul 28 12:43:47.496981 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.496960    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lf2q5"]
Jul 28 12:43:47.587824 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.587790    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24tjj\" (UniqueName: \"kubernetes.io/projected/24e248eb-b35f-4045-9243-6f738428cbc5-kube-api-access-24tjj\") pod \"authorino-5d4b67c446-lf2q5\" (UID: \"24e248eb-b35f-4045-9243-6f738428cbc5\") " pod="kuadrant-system/authorino-5d4b67c446-lf2q5"
Jul 28 12:43:47.647812 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.647778    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lf2q5"]
Jul 28 12:43:47.648076 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:43:47.648056    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-24tjj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-lf2q5" podUID="24e248eb-b35f-4045-9243-6f738428cbc5"
Jul 28 12:43:47.680895 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.680861    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-s7xpw"]
Jul 28 12:43:47.684246 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.684229    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:47.687302 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.687282    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 12:43:47.688386 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.688364    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24tjj\" (UniqueName: \"kubernetes.io/projected/24e248eb-b35f-4045-9243-6f738428cbc5-kube-api-access-24tjj\") pod \"authorino-5d4b67c446-lf2q5\" (UID: \"24e248eb-b35f-4045-9243-6f738428cbc5\") " pod="kuadrant-system/authorino-5d4b67c446-lf2q5"
Jul 28 12:43:47.694016 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.693991    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-s7xpw"]
Jul 28 12:43:47.700947 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.700923    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24tjj\" (UniqueName: \"kubernetes.io/projected/24e248eb-b35f-4045-9243-6f738428cbc5-kube-api-access-24tjj\") pod \"authorino-5d4b67c446-lf2q5\" (UID: \"24e248eb-b35f-4045-9243-6f738428cbc5\") " pod="kuadrant-system/authorino-5d4b67c446-lf2q5"
Jul 28 12:43:47.789095 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.788993    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5rtq\" (UniqueName: \"kubernetes.io/projected/06972752-329f-4623-9fe4-d6489452c192-kube-api-access-k5rtq\") pod \"authorino-76877994-s7xpw\" (UID: \"06972752-329f-4623-9fe4-d6489452c192\") " pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:47.789095 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.789077    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06972752-329f-4623-9fe4-d6489452c192-tls-cert\") pod \"authorino-76877994-s7xpw\" (UID: \"06972752-329f-4623-9fe4-d6489452c192\") " pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:47.793662 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.793618    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s7xpw"]
Jul 28 12:43:47.793904 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:43:47.793886    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-k5rtq tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-s7xpw" podUID="06972752-329f-4623-9fe4-d6489452c192"
Jul 28 12:43:47.817083 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.817044    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:47.817260 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.817044    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lf2q5"
Jul 28 12:43:47.821006 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.820983    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67fc944555-hkkvv"]
Jul 28 12:43:47.824422 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.824403    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:43:47.825169 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.825154    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:47.828293 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.828273    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lf2q5"
Jul 28 12:43:47.831424 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.831399    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67fc944555-hkkvv"]
Jul 28 12:43:47.889435 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.889407    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24tjj\" (UniqueName: \"kubernetes.io/projected/24e248eb-b35f-4045-9243-6f738428cbc5-kube-api-access-24tjj\") pod \"24e248eb-b35f-4045-9243-6f738428cbc5\" (UID: \"24e248eb-b35f-4045-9243-6f738428cbc5\") "
Jul 28 12:43:47.889661 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.889614    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06972752-329f-4623-9fe4-d6489452c192-tls-cert\") pod \"authorino-76877994-s7xpw\" (UID: \"06972752-329f-4623-9fe4-d6489452c192\") " pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:47.889740 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.889720    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w28jh\" (UniqueName: \"kubernetes.io/projected/5e8af227-46f4-4ac5-89b3-5801e770fe7c-kube-api-access-w28jh\") pod \"authorino-67fc944555-hkkvv\" (UID: \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\") " pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:43:47.889797 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.889760    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e8af227-46f4-4ac5-89b3-5801e770fe7c-tls-cert\") pod \"authorino-67fc944555-hkkvv\" (UID: \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\") " pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:43:47.889852 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.889821    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5rtq\" (UniqueName: \"kubernetes.io/projected/06972752-329f-4623-9fe4-d6489452c192-kube-api-access-k5rtq\") pod \"authorino-76877994-s7xpw\" (UID: \"06972752-329f-4623-9fe4-d6489452c192\") " pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:47.891802 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.891769    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24e248eb-b35f-4045-9243-6f738428cbc5-kube-api-access-24tjj" (OuterVolumeSpecName: "kube-api-access-24tjj") pod "24e248eb-b35f-4045-9243-6f738428cbc5" (UID: "24e248eb-b35f-4045-9243-6f738428cbc5"). InnerVolumeSpecName "kube-api-access-24tjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:43:47.892105 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.892089    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06972752-329f-4623-9fe4-d6489452c192-tls-cert\") pod \"authorino-76877994-s7xpw\" (UID: \"06972752-329f-4623-9fe4-d6489452c192\") " pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:47.899523 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.899504    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5rtq\" (UniqueName: \"kubernetes.io/projected/06972752-329f-4623-9fe4-d6489452c192-kube-api-access-k5rtq\") pod \"authorino-76877994-s7xpw\" (UID: \"06972752-329f-4623-9fe4-d6489452c192\") " pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:47.990596 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.990557    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06972752-329f-4623-9fe4-d6489452c192-tls-cert\") pod \"06972752-329f-4623-9fe4-d6489452c192\" (UID: \"06972752-329f-4623-9fe4-d6489452c192\") "
Jul 28 12:43:47.990793 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.990696    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5rtq\" (UniqueName: \"kubernetes.io/projected/06972752-329f-4623-9fe4-d6489452c192-kube-api-access-k5rtq\") pod \"06972752-329f-4623-9fe4-d6489452c192\" (UID: \"06972752-329f-4623-9fe4-d6489452c192\") "
Jul 28 12:43:47.990866 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.990851    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28jh\" (UniqueName: \"kubernetes.io/projected/5e8af227-46f4-4ac5-89b3-5801e770fe7c-kube-api-access-w28jh\") pod \"authorino-67fc944555-hkkvv\" (UID: \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\") " pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:43:47.990925 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.990880    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e8af227-46f4-4ac5-89b3-5801e770fe7c-tls-cert\") pod \"authorino-67fc944555-hkkvv\" (UID: \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\") " pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:43:47.990925 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.990919    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24tjj\" (UniqueName: \"kubernetes.io/projected/24e248eb-b35f-4045-9243-6f738428cbc5-kube-api-access-24tjj\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:43:47.992859 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.992820    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06972752-329f-4623-9fe4-d6489452c192-kube-api-access-k5rtq" (OuterVolumeSpecName: "kube-api-access-k5rtq") pod "06972752-329f-4623-9fe4-d6489452c192" (UID: "06972752-329f-4623-9fe4-d6489452c192"). InnerVolumeSpecName "kube-api-access-k5rtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:43:47.992859 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.992841    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06972752-329f-4623-9fe4-d6489452c192-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "06972752-329f-4623-9fe4-d6489452c192" (UID: "06972752-329f-4623-9fe4-d6489452c192"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:43:47.993271 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.993254    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e8af227-46f4-4ac5-89b3-5801e770fe7c-tls-cert\") pod \"authorino-67fc944555-hkkvv\" (UID: \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\") " pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:43:47.998807 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:47.998790    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w28jh\" (UniqueName: \"kubernetes.io/projected/5e8af227-46f4-4ac5-89b3-5801e770fe7c-kube-api-access-w28jh\") pod \"authorino-67fc944555-hkkvv\" (UID: \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\") " pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:43:48.091965 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.091932    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5rtq\" (UniqueName: \"kubernetes.io/projected/06972752-329f-4623-9fe4-d6489452c192-kube-api-access-k5rtq\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:43:48.091965 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.091960    2549 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/06972752-329f-4623-9fe4-d6489452c192-tls-cert\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:43:48.134807 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.134779    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:43:48.265431 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.265398    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67fc944555-hkkvv"]
Jul 28 12:43:48.268737 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:43:48.268710    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e8af227_46f4_4ac5_89b3_5801e770fe7c.slice/crio-d3f88a8167817b31af179075e598ca71098aa81b6d942545f0d7f1cd2a5b7d0e WatchSource:0}: Error finding container d3f88a8167817b31af179075e598ca71098aa81b6d942545f0d7f1cd2a5b7d0e: Status 404 returned error can't find the container with id d3f88a8167817b31af179075e598ca71098aa81b6d942545f0d7f1cd2a5b7d0e
Jul 28 12:43:48.822418 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.822389    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lf2q5"
Jul 28 12:43:48.822808 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.822423    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67fc944555-hkkvv" event={"ID":"5e8af227-46f4-4ac5-89b3-5801e770fe7c","Type":"ContainerStarted","Data":"d3f88a8167817b31af179075e598ca71098aa81b6d942545f0d7f1cd2a5b7d0e"}
Jul 28 12:43:48.822808 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.822649    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s7xpw"
Jul 28 12:43:48.850999 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.850966    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lf2q5"]
Jul 28 12:43:48.863866 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.863839    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lf2q5"]
Jul 28 12:43:48.883347 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.883322    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s7xpw"]
Jul 28 12:43:48.887858 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:48.887834    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-s7xpw"]
Jul 28 12:43:49.828236 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:49.828199    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67fc944555-hkkvv" event={"ID":"5e8af227-46f4-4ac5-89b3-5801e770fe7c","Type":"ContainerStarted","Data":"5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0"}
Jul 28 12:43:49.844869 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:49.844825    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67fc944555-hkkvv" podStartSLOduration=2.038246623 podStartE2EDuration="2.844811546s" totalImagesPullingTime="806.564923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:43:49.844409245 +0000 UTC m=+821.708659100" watchObservedRunningTime="2026-07-28 12:43:49.844811546 +0000 UTC m=+821.709061405"
Jul 28 12:43:50.692005 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:50.691971    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06972752-329f-4623-9fe4-d6489452c192" path="/var/lib/kubelet/pods/06972752-329f-4623-9fe4-d6489452c192/volumes"
Jul 28 12:43:50.692279 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:50.692262    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24e248eb-b35f-4045-9243-6f738428cbc5" path="/var/lib/kubelet/pods/24e248eb-b35f-4045-9243-6f738428cbc5/volumes"
Jul 28 12:43:55.368030 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.367995    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d844fcccb-zc9sq"]
Jul 28 12:43:55.372586 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.372554    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:43:55.375573 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.375549    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 12:43:55.375703 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.375660    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9jp9d\""
Jul 28 12:43:55.376944 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.376924    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 12:43:55.384303 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.384280    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d844fcccb-zc9sq"]
Jul 28 12:43:55.455361 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.455320    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-webhook-cert\") pod \"maas-controller-6d844fcccb-zc9sq\" (UID: \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\") " pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:43:55.455361 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.455364    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qt54\" (UniqueName: \"kubernetes.io/projected/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-kube-api-access-7qt54\") pod \"maas-controller-6d844fcccb-zc9sq\" (UID: \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\") " pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:43:55.556745 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.556698    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-webhook-cert\") pod \"maas-controller-6d844fcccb-zc9sq\" (UID: \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\") " pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:43:55.556745 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.556746    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qt54\" (UniqueName: \"kubernetes.io/projected/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-kube-api-access-7qt54\") pod \"maas-controller-6d844fcccb-zc9sq\" (UID: \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\") " pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:43:55.559507 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.559481    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-webhook-cert\") pod \"maas-controller-6d844fcccb-zc9sq\" (UID: \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\") " pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:43:55.562050 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.562028    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-nq5r7"]
Jul 28 12:43:55.566081 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.566064    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:43:55.568854 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.568808    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qt54\" (UniqueName: \"kubernetes.io/projected/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-kube-api-access-7qt54\") pod \"maas-controller-6d844fcccb-zc9sq\" (UID: \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\") " pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:43:55.577086 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.577059    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-nq5r7"]
Jul 28 12:43:55.658212 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.658117    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-webhook-cert\") pod \"maas-controller-65dd8bc54b-nq5r7\" (UID: \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\") " pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:43:55.658212 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.658198    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzbqw\" (UniqueName: \"kubernetes.io/projected/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-kube-api-access-bzbqw\") pod \"maas-controller-65dd8bc54b-nq5r7\" (UID: \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\") " pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:43:55.683761 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.683734    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:43:55.759030 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.758997    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzbqw\" (UniqueName: \"kubernetes.io/projected/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-kube-api-access-bzbqw\") pod \"maas-controller-65dd8bc54b-nq5r7\" (UID: \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\") " pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:43:55.759185 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.759084    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-webhook-cert\") pod \"maas-controller-65dd8bc54b-nq5r7\" (UID: \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\") " pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:43:55.761398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.761374    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-webhook-cert\") pod \"maas-controller-65dd8bc54b-nq5r7\" (UID: \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\") " pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:43:55.768963 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.768939    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzbqw\" (UniqueName: \"kubernetes.io/projected/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-kube-api-access-bzbqw\") pod \"maas-controller-65dd8bc54b-nq5r7\" (UID: \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\") " pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:43:55.824200 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.824175    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d844fcccb-zc9sq"]
Jul 28 12:43:55.826616 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:43:55.826583    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c999f47_e0a5_4a2d_a010_d29cdf5bb454.slice/crio-c55e51226b2d01531d30b97fb6a84104b8c5e7f467d65c9a2b2d8f63fbbdc722 WatchSource:0}: Error finding container c55e51226b2d01531d30b97fb6a84104b8c5e7f467d65c9a2b2d8f63fbbdc722: Status 404 returned error can't find the container with id c55e51226b2d01531d30b97fb6a84104b8c5e7f467d65c9a2b2d8f63fbbdc722
Jul 28 12:43:55.855879 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.855846    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d844fcccb-zc9sq" event={"ID":"3c999f47-e0a5-4a2d-a010-d29cdf5bb454","Type":"ContainerStarted","Data":"c55e51226b2d01531d30b97fb6a84104b8c5e7f467d65c9a2b2d8f63fbbdc722"}
Jul 28 12:43:55.888269 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:55.888242    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:43:56.021940 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:56.021913    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-nq5r7"]
Jul 28 12:43:56.024214 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:43:56.024189    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a888e8_3ff1_4d8c_ad3b_04482d4be2a2.slice/crio-4753203ae8cdd628b2bed93e699c3a6eb689ddc43f1dced86ed00d90d44ad858 WatchSource:0}: Error finding container 4753203ae8cdd628b2bed93e699c3a6eb689ddc43f1dced86ed00d90d44ad858: Status 404 returned error can't find the container with id 4753203ae8cdd628b2bed93e699c3a6eb689ddc43f1dced86ed00d90d44ad858
Jul 28 12:43:56.865420 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:56.865378    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7" event={"ID":"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2","Type":"ContainerStarted","Data":"4753203ae8cdd628b2bed93e699c3a6eb689ddc43f1dced86ed00d90d44ad858"}
Jul 28 12:43:58.876544 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:58.876507    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d844fcccb-zc9sq" event={"ID":"3c999f47-e0a5-4a2d-a010-d29cdf5bb454","Type":"ContainerStarted","Data":"4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2"}
Jul 28 12:43:58.877104 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:58.876569    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:43:58.878010 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:58.877985    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7" event={"ID":"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2","Type":"ContainerStarted","Data":"75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712"}
Jul 28 12:43:58.878122 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:58.878102    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:43:58.894568 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:58.894524    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d844fcccb-zc9sq" podStartSLOduration=1.370429973 podStartE2EDuration="3.894512124s" totalImagesPullingTime="2.524082151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:43:58.892888819 +0000 UTC m=+830.757138695" watchObservedRunningTime="2026-07-28 12:43:58.894512124 +0000 UTC m=+830.758761982"
Jul 28 12:43:58.913227 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:43:58.913186    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7" podStartSLOduration=1.579632871 podStartE2EDuration="3.913173582s" totalImagesPullingTime="2.333540711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:43:58.912312088 +0000 UTC m=+830.776561948" watchObservedRunningTime="2026-07-28 12:43:58.913173582 +0000 UTC m=+830.777423441"
Jul 28 12:44:09.884462 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:09.884428    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:44:09.884864 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:09.884511    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:44:09.940353 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:09.940323    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d844fcccb-zc9sq"]
Jul 28 12:44:09.940562 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:09.940514    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d844fcccb-zc9sq" podUID="3c999f47-e0a5-4a2d-a010-d29cdf5bb454" containerName="manager" containerID="cri-o://4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2" gracePeriod=30
Jul 28 12:44:10.202513 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.202486    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:44:10.397573 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.397528    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-webhook-cert\") pod \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\" (UID: \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\") "
Jul 28 12:44:10.397834 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.397664    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qt54\" (UniqueName: \"kubernetes.io/projected/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-kube-api-access-7qt54\") pod \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\" (UID: \"3c999f47-e0a5-4a2d-a010-d29cdf5bb454\") "
Jul 28 12:44:10.399914 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.399885    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3c999f47-e0a5-4a2d-a010-d29cdf5bb454" (UID: "3c999f47-e0a5-4a2d-a010-d29cdf5bb454"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:44:10.399914 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.399902    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-kube-api-access-7qt54" (OuterVolumeSpecName: "kube-api-access-7qt54") pod "3c999f47-e0a5-4a2d-a010-d29cdf5bb454" (UID: "3c999f47-e0a5-4a2d-a010-d29cdf5bb454"). InnerVolumeSpecName "kube-api-access-7qt54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:44:10.499189 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.499110    2549 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-webhook-cert\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:44:10.499189 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.499137    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qt54\" (UniqueName: \"kubernetes.io/projected/3c999f47-e0a5-4a2d-a010-d29cdf5bb454-kube-api-access-7qt54\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:44:10.936819 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.936785    2549 generic.go:356] "Generic (PLEG): container finished" podID="3c999f47-e0a5-4a2d-a010-d29cdf5bb454" containerID="4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2" exitCode=0
Jul 28 12:44:10.937255 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.936831    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d844fcccb-zc9sq" event={"ID":"3c999f47-e0a5-4a2d-a010-d29cdf5bb454","Type":"ContainerDied","Data":"4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2"}
Jul 28 12:44:10.937255 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.936835    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d844fcccb-zc9sq"
Jul 28 12:44:10.937255 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.936874    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d844fcccb-zc9sq" event={"ID":"3c999f47-e0a5-4a2d-a010-d29cdf5bb454","Type":"ContainerDied","Data":"c55e51226b2d01531d30b97fb6a84104b8c5e7f467d65c9a2b2d8f63fbbdc722"}
Jul 28 12:44:10.937255 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.936898    2549 scope.go:117] "RemoveContainer" containerID="4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2"
Jul 28 12:44:10.945438 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.945417    2549 scope.go:117] "RemoveContainer" containerID="4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2"
Jul 28 12:44:10.945810 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:44:10.945789    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2\": container with ID starting with 4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2 not found: ID does not exist" containerID="4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2"
Jul 28 12:44:10.945864 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.945818    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2"} err="failed to get container status \"4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2\": rpc error: code = NotFound desc = could not find container \"4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2\": container with ID starting with 4aef572ce64d5fd53b53ea587a2e5c578b7d8fc1fabf0dcc0feb977b63b1cab2 not found: ID does not exist"
Jul 28 12:44:10.954184 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.954164    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d844fcccb-zc9sq"]
Jul 28 12:44:10.960096 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:10.960074    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d844fcccb-zc9sq"]
Jul 28 12:44:12.691792 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:12.691759    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c999f47-e0a5-4a2d-a010-d29cdf5bb454" path="/var/lib/kubelet/pods/3c999f47-e0a5-4a2d-a010-d29cdf5bb454/volumes"
Jul 28 12:44:31.992280 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:31.992243    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8649546779-m2tj8"]
Jul 28 12:44:31.992725 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:31.992651    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c999f47-e0a5-4a2d-a010-d29cdf5bb454" containerName="manager"
Jul 28 12:44:31.992725 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:31.992665    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c999f47-e0a5-4a2d-a010-d29cdf5bb454" containerName="manager"
Jul 28 12:44:31.992797 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:31.992737    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c999f47-e0a5-4a2d-a010-d29cdf5bb454" containerName="manager"
Jul 28 12:44:31.995741 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:31.995720    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:31.999231 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:31.999209    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 12:44:32.000733 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.000714    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d58lq\""
Jul 28 12:44:32.006224 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.006198    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8649546779-m2tj8"]
Jul 28 12:44:32.064193 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.064165    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/275f404d-e226-45f8-a411-d86b8b6d2963-openshift-service-ca\") pod \"maas-api-8649546779-m2tj8\" (UID: \"275f404d-e226-45f8-a411-d86b8b6d2963\") " pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.064396 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.064221    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/275f404d-e226-45f8-a411-d86b8b6d2963-maas-api-tls\") pod \"maas-api-8649546779-m2tj8\" (UID: \"275f404d-e226-45f8-a411-d86b8b6d2963\") " pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.064396 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.064279    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mkdz\" (UniqueName: \"kubernetes.io/projected/275f404d-e226-45f8-a411-d86b8b6d2963-kube-api-access-7mkdz\") pod \"maas-api-8649546779-m2tj8\" (UID: \"275f404d-e226-45f8-a411-d86b8b6d2963\") " pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.165459 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.165423    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/275f404d-e226-45f8-a411-d86b8b6d2963-maas-api-tls\") pod \"maas-api-8649546779-m2tj8\" (UID: \"275f404d-e226-45f8-a411-d86b8b6d2963\") " pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.165672 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.165484    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mkdz\" (UniqueName: \"kubernetes.io/projected/275f404d-e226-45f8-a411-d86b8b6d2963-kube-api-access-7mkdz\") pod \"maas-api-8649546779-m2tj8\" (UID: \"275f404d-e226-45f8-a411-d86b8b6d2963\") " pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.165672 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.165564    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/275f404d-e226-45f8-a411-d86b8b6d2963-openshift-service-ca\") pod \"maas-api-8649546779-m2tj8\" (UID: \"275f404d-e226-45f8-a411-d86b8b6d2963\") " pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.166147 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.166118    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/275f404d-e226-45f8-a411-d86b8b6d2963-openshift-service-ca\") pod \"maas-api-8649546779-m2tj8\" (UID: \"275f404d-e226-45f8-a411-d86b8b6d2963\") " pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.167797 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.167778    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/275f404d-e226-45f8-a411-d86b8b6d2963-maas-api-tls\") pod \"maas-api-8649546779-m2tj8\" (UID: \"275f404d-e226-45f8-a411-d86b8b6d2963\") " pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.175063 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.175038    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mkdz\" (UniqueName: \"kubernetes.io/projected/275f404d-e226-45f8-a411-d86b8b6d2963-kube-api-access-7mkdz\") pod \"maas-api-8649546779-m2tj8\" (UID: \"275f404d-e226-45f8-a411-d86b8b6d2963\") " pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.306887 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.306850    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:32.470737 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.470705    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8649546779-m2tj8"]
Jul 28 12:44:32.473191 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:44:32.473163    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275f404d_e226_45f8_a411_d86b8b6d2963.slice/crio-d25b4054f67978cfcaeae8a865d796249dd5767d626e46488625a59f7bbac02a WatchSource:0}: Error finding container d25b4054f67978cfcaeae8a865d796249dd5767d626e46488625a59f7bbac02a: Status 404 returned error can't find the container with id d25b4054f67978cfcaeae8a865d796249dd5767d626e46488625a59f7bbac02a
Jul 28 12:44:32.630821 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.630741    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"]
Jul 28 12:44:32.635278 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.635257    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:44:32.638759 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.638732    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 28 12:44:32.650933 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.650912    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"]
Jul 28 12:44:32.670289 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.670265    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7bfe86a-b217-4873-88b7-052fd8b09ad2-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-r4f55\" (UID: \"a7bfe86a-b217-4873-88b7-052fd8b09ad2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:44:32.670421 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.670307    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs9ss\" (UniqueName: \"kubernetes.io/projected/a7bfe86a-b217-4873-88b7-052fd8b09ad2-kube-api-access-qs9ss\") pod \"payload-pre-processing-687dbb6cf6-r4f55\" (UID: \"a7bfe86a-b217-4873-88b7-052fd8b09ad2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:44:32.771302 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.771262    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7bfe86a-b217-4873-88b7-052fd8b09ad2-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-r4f55\" (UID: \"a7bfe86a-b217-4873-88b7-052fd8b09ad2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:44:32.771488 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.771334    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs9ss\" (UniqueName: \"kubernetes.io/projected/a7bfe86a-b217-4873-88b7-052fd8b09ad2-kube-api-access-qs9ss\") pod \"payload-pre-processing-687dbb6cf6-r4f55\" (UID: \"a7bfe86a-b217-4873-88b7-052fd8b09ad2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:44:32.772234 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.772206    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7bfe86a-b217-4873-88b7-052fd8b09ad2-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-r4f55\" (UID: \"a7bfe86a-b217-4873-88b7-052fd8b09ad2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:44:32.782700 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.782671    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs9ss\" (UniqueName: \"kubernetes.io/projected/a7bfe86a-b217-4873-88b7-052fd8b09ad2-kube-api-access-qs9ss\") pod \"payload-pre-processing-687dbb6cf6-r4f55\" (UID: \"a7bfe86a-b217-4873-88b7-052fd8b09ad2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:44:32.946707 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:32.946601    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:44:33.047038 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:33.046997    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8" event={"ID":"275f404d-e226-45f8-a411-d86b8b6d2963","Type":"ContainerStarted","Data":"d25b4054f67978cfcaeae8a865d796249dd5767d626e46488625a59f7bbac02a"}
Jul 28 12:44:33.103745 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:33.103717    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"]
Jul 28 12:44:33.105852 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:44:33.105829    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7bfe86a_b217_4873_88b7_052fd8b09ad2.slice/crio-649c7ff1c46a86857967c92c66ae478dba70171e327ce4fa1f716b77dc08119a WatchSource:0}: Error finding container 649c7ff1c46a86857967c92c66ae478dba70171e327ce4fa1f716b77dc08119a: Status 404 returned error can't find the container with id 649c7ff1c46a86857967c92c66ae478dba70171e327ce4fa1f716b77dc08119a
Jul 28 12:44:34.058406 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:34.058366    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55" event={"ID":"a7bfe86a-b217-4873-88b7-052fd8b09ad2","Type":"ContainerStarted","Data":"649c7ff1c46a86857967c92c66ae478dba70171e327ce4fa1f716b77dc08119a"}
Jul 28 12:44:35.067260 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:35.066920    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8" event={"ID":"275f404d-e226-45f8-a411-d86b8b6d2963","Type":"ContainerStarted","Data":"d2c03bd17e6a7d17232eb2b01619b2392cfb7a97188504d1a51ce410491344e9"}
Jul 28 12:44:35.067260 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:35.067070    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:36.076047 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:36.076006    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55" event={"ID":"a7bfe86a-b217-4873-88b7-052fd8b09ad2","Type":"ContainerStarted","Data":"c4054c58464bead1415d132de1b2955ba4c632129a8bd6037d2e7c9cd77d60b1"}
Jul 28 12:44:36.076564 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:36.076097    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:44:36.096148 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:36.096090    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55" podStartSLOduration=2.019384087 podStartE2EDuration="4.096066345s" totalImagesPullingTime="2.076682258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:44:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:44:36.094865955 +0000 UTC m=+867.959115827" watchObservedRunningTime="2026-07-28 12:44:36.096066345 +0000 UTC m=+867.960316207"
Jul 28 12:44:36.096749 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:36.096709    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8" podStartSLOduration=2.97623365 podStartE2EDuration="5.096695368s" totalImagesPullingTime="2.120461718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:44:35.100254118 +0000 UTC m=+866.964503979" watchObservedRunningTime="2026-07-28 12:44:36.096695368 +0000 UTC m=+867.960945228"
Jul 28 12:44:41.080730 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:41.080698    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8649546779-m2tj8"
Jul 28 12:44:41.103056 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:41.103027    2549 generic.go:356] "Generic (PLEG): container finished" podID="db993d21-5f17-4559-94b3-075a8a6acac2" containerID="d1b6faf3ac7c37827de4a8a6513c6ccedd6984446dbd9598055ee476b79bdafd" exitCode=0
Jul 28 12:44:41.103320 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:41.103202    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" event={"ID":"db993d21-5f17-4559-94b3-075a8a6acac2","Type":"ContainerDied","Data":"d1b6faf3ac7c37827de4a8a6513c6ccedd6984446dbd9598055ee476b79bdafd"}
Jul 28 12:44:41.103781 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:41.103763    2549 scope.go:117] "RemoveContainer" containerID="d1b6faf3ac7c37827de4a8a6513c6ccedd6984446dbd9598055ee476b79bdafd"
Jul 28 12:44:41.104857 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:41.104832    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:44:41.104950 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:41.104891    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:44:41.104950 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:41.104923    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 12:44:42.109887 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:42.109850    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" event={"ID":"db993d21-5f17-4559-94b3-075a8a6acac2","Type":"ContainerStarted","Data":"503e50daae1e5cc2089420e1fc00a427bac73108afcd1c433ae71b1ea7c2f14f"}
Jul 28 12:44:42.144385 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:42.144352    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:44:42.145305 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:42.145280    2549 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-j4qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 28 12:44:42.145410 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:42.145338    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" podUID="db993d21-5f17-4559-94b3-075a8a6acac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 28 12:44:42.576498 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:42.576435    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:44:43.144184 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:43.144152    2549 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-j4qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 28 12:44:43.144562 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:43.144209    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" podUID="db993d21-5f17-4559-94b3-075a8a6acac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 28 12:44:44.144327 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:44.144296    2549 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-j4qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 28 12:44:44.144741 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:44.144362    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" podUID="db993d21-5f17-4559-94b3-075a8a6acac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 28 12:44:45.144511 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:45.144480    2549 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-j4qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 28 12:44:45.144904 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:45.144532    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" podUID="db993d21-5f17-4559-94b3-075a8a6acac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 28 12:44:46.143748 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:46.143712    2549 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-j4qxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 28 12:44:46.143933 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:46.143764    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs" podUID="db993d21-5f17-4559-94b3-075a8a6acac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 28 12:44:47.148203 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:47.148173    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:44:47.149200 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:47.149180    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j4qxs"
Jul 28 12:44:48.088741 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:44:48.088710    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-r4f55"
Jul 28 12:45:00.166019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:00.165927    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5"]
Jul 28 12:45:00.170816 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:00.170792    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5"
Jul 28 12:45:00.177083 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:00.177056    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j5hqn\""
Jul 28 12:45:00.187995 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:00.187970    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5"]
Jul 28 12:45:00.196461 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:00.196429    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5"
Jul 28 12:45:00.340016 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:00.339985    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5"]
Jul 28 12:45:00.345357 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:45:00.345323    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3b9ade_1fa4_412f_b9f7_6a6f838e87f9.slice/crio-a0749d5c33a682b34ff92d9fa34a5a7e08b5f58303c607d63fe2e61e61356144 WatchSource:0}: Error finding container a0749d5c33a682b34ff92d9fa34a5a7e08b5f58303c607d63fe2e61e61356144: Status 404 returned error can't find the container with id a0749d5c33a682b34ff92d9fa34a5a7e08b5f58303c607d63fe2e61e61356144
Jul 28 12:45:01.202534 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:01.202493    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5" event={"ID":"3f3b9ade-1fa4-412f-b9f7-6a6f838e87f9","Type":"ContainerStarted","Data":"a0749d5c33a682b34ff92d9fa34a5a7e08b5f58303c607d63fe2e61e61356144"}
Jul 28 12:45:05.225961 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:05.225918    2549 generic.go:356] "Generic (PLEG): container finished" podID="3f3b9ade-1fa4-412f-b9f7-6a6f838e87f9" containerID="6e85d12e23751973682141d32fd8c683f9c4db9e6d01f4d8b90fbc6ebf7e3f28" exitCode=0
Jul 28 12:45:05.226354 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:05.225966    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5" event={"ID":"3f3b9ade-1fa4-412f-b9f7-6a6f838e87f9","Type":"ContainerDied","Data":"6e85d12e23751973682141d32fd8c683f9c4db9e6d01f4d8b90fbc6ebf7e3f28"}
Jul 28 12:45:06.365842 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:06.365812    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5"
Jul 28 12:45:07.237924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:07.237893    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5"
Jul 28 12:45:07.237924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:07.237909    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754045-gvzx5" event={"ID":"3f3b9ade-1fa4-412f-b9f7-6a6f838e87f9","Type":"ContainerDied","Data":"a0749d5c33a682b34ff92d9fa34a5a7e08b5f58303c607d63fe2e61e61356144"}
Jul 28 12:45:07.238183 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:07.237950    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0749d5c33a682b34ff92d9fa34a5a7e08b5f58303c607d63fe2e61e61356144"
Jul 28 12:45:08.582552 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:08.582515    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:45:08.583028 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:08.582821    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:45:13.626653 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.626591    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"]
Jul 28 12:45:13.627038 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.626983    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f3b9ade-1fa4-412f-b9f7-6a6f838e87f9" containerName="cleanup"
Jul 28 12:45:13.627038 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.626994    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f3b9ade-1fa4-412f-b9f7-6a6f838e87f9" containerName="cleanup"
Jul 28 12:45:13.627113 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.627065    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f3b9ade-1fa4-412f-b9f7-6a6f838e87f9" containerName="cleanup"
Jul 28 12:45:13.631393 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.631377    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.634924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.634898    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 12:45:13.634924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.634909    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 12:45:13.635179 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.635164    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p75qd\""
Jul 28 12:45:13.635358 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.635345    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 12:45:13.645568 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.645541    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"]
Jul 28 12:45:13.652142 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.652117    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.652275 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.652170    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.652275 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.652234    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b00dec-0ac2-4cec-8f1e-478cf0497327-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.652400 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.652341    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.652459 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.652399    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.652459 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.652443    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlfdn\" (UniqueName: \"kubernetes.io/projected/d3b00dec-0ac2-4cec-8f1e-478cf0497327-kube-api-access-rlfdn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.753520 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.753485    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.753520 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.753523    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.753796 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.753542    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlfdn\" (UniqueName: \"kubernetes.io/projected/d3b00dec-0ac2-4cec-8f1e-478cf0497327-kube-api-access-rlfdn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.753796 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.753572    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.753796 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.753608    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.753796 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.753659    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b00dec-0ac2-4cec-8f1e-478cf0497327-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.753999 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.753934    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.754039 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.754016    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.754073 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.754056    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.756044 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.756020    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b00dec-0ac2-4cec-8f1e-478cf0497327-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.756144 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.756109    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b00dec-0ac2-4cec-8f1e-478cf0497327-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.765248 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.765226    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlfdn\" (UniqueName: \"kubernetes.io/projected/d3b00dec-0ac2-4cec-8f1e-478cf0497327-kube-api-access-rlfdn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2rfmw\" (UID: \"d3b00dec-0ac2-4cec-8f1e-478cf0497327\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:13.942572 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:13.942492    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:14.080575 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:14.080547    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"]
Jul 28 12:45:14.083155 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:45:14.083127    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b00dec_0ac2_4cec_8f1e_478cf0497327.slice/crio-313c1721a788ec14778fc4dce31761079b5e6ef1c132b1ab1603c32113dfde9e WatchSource:0}: Error finding container 313c1721a788ec14778fc4dce31761079b5e6ef1c132b1ab1603c32113dfde9e: Status 404 returned error can't find the container with id 313c1721a788ec14778fc4dce31761079b5e6ef1c132b1ab1603c32113dfde9e
Jul 28 12:45:14.274064 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:14.273976    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw" event={"ID":"d3b00dec-0ac2-4cec-8f1e-478cf0497327","Type":"ContainerStarted","Data":"313c1721a788ec14778fc4dce31761079b5e6ef1c132b1ab1603c32113dfde9e"}
Jul 28 12:45:15.280642 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:15.280594    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw" event={"ID":"d3b00dec-0ac2-4cec-8f1e-478cf0497327","Type":"ContainerStarted","Data":"9525538b7761ac87c7a0cb710bee6e7290aea160bd3e551311ef9a0ff089169b"}
Jul 28 12:45:15.301822 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:15.301772    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw" podStartSLOduration=1.254741932 podStartE2EDuration="2.301752325s" totalImagesPullingTime="1.047010393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:45:15.299504361 +0000 UTC m=+907.163754221" watchObservedRunningTime="2026-07-28 12:45:15.301752325 +0000 UTC m=+907.166002189"
Jul 28 12:45:23.943563 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:23.943523    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:23.944076 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:23.943609    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:23.956492 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:23.956465    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:45:24.335681 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:45:24.335647    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2rfmw"
Jul 28 12:46:33.362537 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.362460    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74d7594bf4-xc4ht"]
Jul 28 12:46:33.366094 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.366075    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74d7594bf4-xc4ht"
Jul 28 12:46:33.375996 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.375971    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74d7594bf4-xc4ht"]
Jul 28 12:46:33.482306 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.482266    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea856dd5-8ac0-476b-87bf-5abb684f3cdc-tls-cert\") pod \"authorino-74d7594bf4-xc4ht\" (UID: \"ea856dd5-8ac0-476b-87bf-5abb684f3cdc\") " pod="kuadrant-system/authorino-74d7594bf4-xc4ht"
Jul 28 12:46:33.482306 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.482311    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbk48\" (UniqueName: \"kubernetes.io/projected/ea856dd5-8ac0-476b-87bf-5abb684f3cdc-kube-api-access-vbk48\") pod \"authorino-74d7594bf4-xc4ht\" (UID: \"ea856dd5-8ac0-476b-87bf-5abb684f3cdc\") " pod="kuadrant-system/authorino-74d7594bf4-xc4ht"
Jul 28 12:46:33.583254 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.583214    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea856dd5-8ac0-476b-87bf-5abb684f3cdc-tls-cert\") pod \"authorino-74d7594bf4-xc4ht\" (UID: \"ea856dd5-8ac0-476b-87bf-5abb684f3cdc\") " pod="kuadrant-system/authorino-74d7594bf4-xc4ht"
Jul 28 12:46:33.583419 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.583264    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbk48\" (UniqueName: \"kubernetes.io/projected/ea856dd5-8ac0-476b-87bf-5abb684f3cdc-kube-api-access-vbk48\") pod \"authorino-74d7594bf4-xc4ht\" (UID: \"ea856dd5-8ac0-476b-87bf-5abb684f3cdc\") " pod="kuadrant-system/authorino-74d7594bf4-xc4ht"
Jul 28 12:46:33.585855 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.585833    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea856dd5-8ac0-476b-87bf-5abb684f3cdc-tls-cert\") pod \"authorino-74d7594bf4-xc4ht\" (UID: \"ea856dd5-8ac0-476b-87bf-5abb684f3cdc\") " pod="kuadrant-system/authorino-74d7594bf4-xc4ht"
Jul 28 12:46:33.594829 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.594806    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbk48\" (UniqueName: \"kubernetes.io/projected/ea856dd5-8ac0-476b-87bf-5abb684f3cdc-kube-api-access-vbk48\") pod \"authorino-74d7594bf4-xc4ht\" (UID: \"ea856dd5-8ac0-476b-87bf-5abb684f3cdc\") " pod="kuadrant-system/authorino-74d7594bf4-xc4ht"
Jul 28 12:46:33.675211 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.675131    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74d7594bf4-xc4ht"
Jul 28 12:46:33.816187 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.816162    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74d7594bf4-xc4ht"]
Jul 28 12:46:33.817968 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:46:33.817938    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea856dd5_8ac0_476b_87bf_5abb684f3cdc.slice/crio-3a0394f5e85b7d21bdc7ae7a694ea8d91ed527181a03192cb77f243f7c65cedf WatchSource:0}: Error finding container 3a0394f5e85b7d21bdc7ae7a694ea8d91ed527181a03192cb77f243f7c65cedf: Status 404 returned error can't find the container with id 3a0394f5e85b7d21bdc7ae7a694ea8d91ed527181a03192cb77f243f7c65cedf
Jul 28 12:46:33.819220 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:33.819204    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 12:46:34.671742 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:34.670963    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74d7594bf4-xc4ht" event={"ID":"ea856dd5-8ac0-476b-87bf-5abb684f3cdc","Type":"ContainerStarted","Data":"afeec82fe5d591976f0701c61dd029d97f404bd6f9743cfad3bea6d0491ac6e6"}
Jul 28 12:46:34.671742 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:34.671014    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74d7594bf4-xc4ht" event={"ID":"ea856dd5-8ac0-476b-87bf-5abb684f3cdc","Type":"ContainerStarted","Data":"3a0394f5e85b7d21bdc7ae7a694ea8d91ed527181a03192cb77f243f7c65cedf"}
Jul 28 12:46:34.693093 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:34.693024    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74d7594bf4-xc4ht" podStartSLOduration=1.313676955 podStartE2EDuration="1.693001808s" totalImagesPullingTime="379.324853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:46:34.690281566 +0000 UTC m=+986.554531423" watchObservedRunningTime="2026-07-28 12:46:34.693001808 +0000 UTC m=+986.557251667"
Jul 28 12:46:34.734181 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:34.734134    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67fc944555-hkkvv"]
Jul 28 12:46:34.734446 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:34.734386    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-67fc944555-hkkvv" podUID="5e8af227-46f4-4ac5-89b3-5801e770fe7c" containerName="authorino" containerID="cri-o://5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0" gracePeriod=30
Jul 28 12:46:34.996684 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:34.996656    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:46:35.096583 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.096546    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w28jh\" (UniqueName: \"kubernetes.io/projected/5e8af227-46f4-4ac5-89b3-5801e770fe7c-kube-api-access-w28jh\") pod \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\" (UID: \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\") "
Jul 28 12:46:35.096797 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.096688    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e8af227-46f4-4ac5-89b3-5801e770fe7c-tls-cert\") pod \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\" (UID: \"5e8af227-46f4-4ac5-89b3-5801e770fe7c\") "
Jul 28 12:46:35.098775 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.098744    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e8af227-46f4-4ac5-89b3-5801e770fe7c-kube-api-access-w28jh" (OuterVolumeSpecName: "kube-api-access-w28jh") pod "5e8af227-46f4-4ac5-89b3-5801e770fe7c" (UID: "5e8af227-46f4-4ac5-89b3-5801e770fe7c"). InnerVolumeSpecName "kube-api-access-w28jh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:46:35.108170 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.108143    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e8af227-46f4-4ac5-89b3-5801e770fe7c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5e8af227-46f4-4ac5-89b3-5801e770fe7c" (UID: "5e8af227-46f4-4ac5-89b3-5801e770fe7c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:46:35.198340 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.198300    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w28jh\" (UniqueName: \"kubernetes.io/projected/5e8af227-46f4-4ac5-89b3-5801e770fe7c-kube-api-access-w28jh\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:46:35.198340 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.198331    2549 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e8af227-46f4-4ac5-89b3-5801e770fe7c-tls-cert\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:46:35.677211 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.677170    2549 generic.go:356] "Generic (PLEG): container finished" podID="5e8af227-46f4-4ac5-89b3-5801e770fe7c" containerID="5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0" exitCode=0
Jul 28 12:46:35.677211 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.677215    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67fc944555-hkkvv"
Jul 28 12:46:35.677725 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.677271    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67fc944555-hkkvv" event={"ID":"5e8af227-46f4-4ac5-89b3-5801e770fe7c","Type":"ContainerDied","Data":"5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0"}
Jul 28 12:46:35.677725 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.677339    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67fc944555-hkkvv" event={"ID":"5e8af227-46f4-4ac5-89b3-5801e770fe7c","Type":"ContainerDied","Data":"d3f88a8167817b31af179075e598ca71098aa81b6d942545f0d7f1cd2a5b7d0e"}
Jul 28 12:46:35.677725 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.677372    2549 scope.go:117] "RemoveContainer" containerID="5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0"
Jul 28 12:46:35.690716 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.690698    2549 scope.go:117] "RemoveContainer" containerID="5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0"
Jul 28 12:46:35.691018 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:46:35.690998    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0\": container with ID starting with 5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0 not found: ID does not exist" containerID="5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0"
Jul 28 12:46:35.691072 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.691028    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0"} err="failed to get container status \"5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0\": rpc error: code = NotFound desc = could not find container \"5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0\": container with ID starting with 5def52c56841e586cbffb983692e4ea5352d0a45384be7d77fec138a3f9d49c0 not found: ID does not exist"
Jul 28 12:46:35.712506 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.712474    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67fc944555-hkkvv"]
Jul 28 12:46:35.729025 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:35.728996    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-67fc944555-hkkvv"]
Jul 28 12:46:36.691404 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:46:36.691373    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e8af227-46f4-4ac5-89b3-5801e770fe7c" path="/var/lib/kubelet/pods/5e8af227-46f4-4ac5-89b3-5801e770fe7c/volumes"
Jul 28 12:47:21.810647 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:21.809876    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-nq5r7"]
Jul 28 12:47:21.811214 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:21.810430    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7" podUID="11a888e8-3ff1-4d8c-ad3b-04482d4be2a2" containerName="manager" containerID="cri-o://75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712" gracePeriod=30
Jul 28 12:47:22.074197 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.074099    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:47:22.144121 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.144089    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzbqw\" (UniqueName: \"kubernetes.io/projected/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-kube-api-access-bzbqw\") pod \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\" (UID: \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\") "
Jul 28 12:47:22.144288 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.144196    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-webhook-cert\") pod \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\" (UID: \"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2\") "
Jul 28 12:47:22.147737 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.147691    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "11a888e8-3ff1-4d8c-ad3b-04482d4be2a2" (UID: "11a888e8-3ff1-4d8c-ad3b-04482d4be2a2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:47:22.147867 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.147741    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-kube-api-access-bzbqw" (OuterVolumeSpecName: "kube-api-access-bzbqw") pod "11a888e8-3ff1-4d8c-ad3b-04482d4be2a2" (UID: "11a888e8-3ff1-4d8c-ad3b-04482d4be2a2"). InnerVolumeSpecName "kube-api-access-bzbqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:47:22.245298 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.245250    2549 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-webhook-cert\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:47:22.245298 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.245283    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzbqw\" (UniqueName: \"kubernetes.io/projected/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2-kube-api-access-bzbqw\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:47:22.917466 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.917428    2549 generic.go:356] "Generic (PLEG): container finished" podID="11a888e8-3ff1-4d8c-ad3b-04482d4be2a2" containerID="75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712" exitCode=0
Jul 28 12:47:22.917960 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.917476    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7"
Jul 28 12:47:22.917960 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.917518    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7" event={"ID":"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2","Type":"ContainerDied","Data":"75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712"}
Jul 28 12:47:22.917960 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.917586    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-nq5r7" event={"ID":"11a888e8-3ff1-4d8c-ad3b-04482d4be2a2","Type":"ContainerDied","Data":"4753203ae8cdd628b2bed93e699c3a6eb689ddc43f1dced86ed00d90d44ad858"}
Jul 28 12:47:22.917960 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.917648    2549 scope.go:117] "RemoveContainer" containerID="75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712"
Jul 28 12:47:22.926773 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.926753    2549 scope.go:117] "RemoveContainer" containerID="75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712"
Jul 28 12:47:22.927023 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:47:22.927004    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712\": container with ID starting with 75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712 not found: ID does not exist" containerID="75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712"
Jul 28 12:47:22.927083 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.927035    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712"} err="failed to get container status \"75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712\": rpc error: code = NotFound desc = could not find container \"75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712\": container with ID starting with 75917ee7590e737e4476b700e142c6df5a591545f7174785ce25656f91087712 not found: ID does not exist"
Jul 28 12:47:22.938053 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.938018    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-nq5r7"]
Jul 28 12:47:22.943563 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:22.943534    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-nq5r7"]
Jul 28 12:47:24.691854 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:47:24.691812    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11a888e8-3ff1-4d8c-ad3b-04482d4be2a2" path="/var/lib/kubelet/pods/11a888e8-3ff1-4d8c-ad3b-04482d4be2a2/volumes"
Jul 28 12:48:23.850573 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.850536    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-jjl5g"]
Jul 28 12:48:23.850990 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.850977    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e8af227-46f4-4ac5-89b3-5801e770fe7c" containerName="authorino"
Jul 28 12:48:23.850990 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.850990    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e8af227-46f4-4ac5-89b3-5801e770fe7c" containerName="authorino"
Jul 28 12:48:23.851060 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.851000    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a888e8-3ff1-4d8c-ad3b-04482d4be2a2" containerName="manager"
Jul 28 12:48:23.851060 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.851006    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a888e8-3ff1-4d8c-ad3b-04482d4be2a2" containerName="manager"
Jul 28 12:48:23.851124 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.851065    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a888e8-3ff1-4d8c-ad3b-04482d4be2a2" containerName="manager"
Jul 28 12:48:23.851124 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.851074    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e8af227-46f4-4ac5-89b3-5801e770fe7c" containerName="authorino"
Jul 28 12:48:23.854238 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.854222    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:23.857501 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.857480    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 12:48:23.857721 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.857704    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9jp9d\""
Jul 28 12:48:23.859127 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.859109    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 12:48:23.867667 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.867614    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-jjl5g"]
Jul 28 12:48:23.994144 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.994104    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rwqw\" (UniqueName: \"kubernetes.io/projected/a4d61102-30e2-4861-a087-65b61421657b-kube-api-access-9rwqw\") pod \"maas-controller-65dd8bc54b-jjl5g\" (UID: \"a4d61102-30e2-4861-a087-65b61421657b\") " pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:23.994331 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:23.994183    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4d61102-30e2-4861-a087-65b61421657b-webhook-cert\") pod \"maas-controller-65dd8bc54b-jjl5g\" (UID: \"a4d61102-30e2-4861-a087-65b61421657b\") " pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:24.094977 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:24.094942    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rwqw\" (UniqueName: \"kubernetes.io/projected/a4d61102-30e2-4861-a087-65b61421657b-kube-api-access-9rwqw\") pod \"maas-controller-65dd8bc54b-jjl5g\" (UID: \"a4d61102-30e2-4861-a087-65b61421657b\") " pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:24.095149 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:24.095124    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4d61102-30e2-4861-a087-65b61421657b-webhook-cert\") pod \"maas-controller-65dd8bc54b-jjl5g\" (UID: \"a4d61102-30e2-4861-a087-65b61421657b\") " pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:24.097680 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:24.097661    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4d61102-30e2-4861-a087-65b61421657b-webhook-cert\") pod \"maas-controller-65dd8bc54b-jjl5g\" (UID: \"a4d61102-30e2-4861-a087-65b61421657b\") " pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:24.105022 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:24.104954    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rwqw\" (UniqueName: \"kubernetes.io/projected/a4d61102-30e2-4861-a087-65b61421657b-kube-api-access-9rwqw\") pod \"maas-controller-65dd8bc54b-jjl5g\" (UID: \"a4d61102-30e2-4861-a087-65b61421657b\") " pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:24.165073 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:24.165036    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:24.300118 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:24.300095    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-jjl5g"]
Jul 28 12:48:24.302078 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:48:24.302053    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d61102_30e2_4861_a087_65b61421657b.slice/crio-a9dbc378d1219b77c8fb5a2a9d49ebb5199d5f87d82e9d9719b1d8e45b73c589 WatchSource:0}: Error finding container a9dbc378d1219b77c8fb5a2a9d49ebb5199d5f87d82e9d9719b1d8e45b73c589: Status 404 returned error can't find the container with id a9dbc378d1219b77c8fb5a2a9d49ebb5199d5f87d82e9d9719b1d8e45b73c589
Jul 28 12:48:25.227924 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:25.227889    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g" event={"ID":"a4d61102-30e2-4861-a087-65b61421657b","Type":"ContainerStarted","Data":"6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930"}
Jul 28 12:48:25.228298 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:25.227942    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g" event={"ID":"a4d61102-30e2-4861-a087-65b61421657b","Type":"ContainerStarted","Data":"a9dbc378d1219b77c8fb5a2a9d49ebb5199d5f87d82e9d9719b1d8e45b73c589"}
Jul 28 12:48:25.228298 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:25.227976    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:25.249341 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:25.249289    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g" podStartSLOduration=1.836818826 podStartE2EDuration="2.249271794s" totalImagesPullingTime="412.452968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:48:25.247591629 +0000 UTC m=+1097.111841489" watchObservedRunningTime="2026-07-28 12:48:25.249271794 +0000 UTC m=+1097.113521659"
Jul 28 12:48:36.236713 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:36.236681    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:40.951586 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:40.951552    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-jjl5g"]
Jul 28 12:48:40.952150 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:40.951739    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g" podUID="a4d61102-30e2-4861-a087-65b61421657b" containerName="manager" containerID="cri-o://6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930" gracePeriod=30
Jul 28 12:48:41.208114 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.208054    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:41.308132 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.308092    2549 generic.go:356] "Generic (PLEG): container finished" podID="a4d61102-30e2-4861-a087-65b61421657b" containerID="6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930" exitCode=0
Jul 28 12:48:41.308291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.308143    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g"
Jul 28 12:48:41.308291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.308175    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g" event={"ID":"a4d61102-30e2-4861-a087-65b61421657b","Type":"ContainerDied","Data":"6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930"}
Jul 28 12:48:41.308291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.308255    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-jjl5g" event={"ID":"a4d61102-30e2-4861-a087-65b61421657b","Type":"ContainerDied","Data":"a9dbc378d1219b77c8fb5a2a9d49ebb5199d5f87d82e9d9719b1d8e45b73c589"}
Jul 28 12:48:41.308291 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.308278    2549 scope.go:117] "RemoveContainer" containerID="6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930"
Jul 28 12:48:41.316898 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.316877    2549 scope.go:117] "RemoveContainer" containerID="6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930"
Jul 28 12:48:41.317145 ip-10-0-132-90 kubenswrapper[2549]: E0728 12:48:41.317125    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930\": container with ID starting with 6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930 not found: ID does not exist" containerID="6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930"
Jul 28 12:48:41.317211 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.317158    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930"} err="failed to get container status \"6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930\": rpc error: code = NotFound desc = could not find container \"6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930\": container with ID starting with 6115903234c5be74c4d8445d4e0e5a8b837f452475b019d79fd498acb2f44930 not found: ID does not exist"
Jul 28 12:48:41.353755 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.353727    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4d61102-30e2-4861-a087-65b61421657b-webhook-cert\") pod \"a4d61102-30e2-4861-a087-65b61421657b\" (UID: \"a4d61102-30e2-4861-a087-65b61421657b\") "
Jul 28 12:48:41.353908 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.353766    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rwqw\" (UniqueName: \"kubernetes.io/projected/a4d61102-30e2-4861-a087-65b61421657b-kube-api-access-9rwqw\") pod \"a4d61102-30e2-4861-a087-65b61421657b\" (UID: \"a4d61102-30e2-4861-a087-65b61421657b\") "
Jul 28 12:48:41.355920 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.355886    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4d61102-30e2-4861-a087-65b61421657b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a4d61102-30e2-4861-a087-65b61421657b" (UID: "a4d61102-30e2-4861-a087-65b61421657b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 12:48:41.356033 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.355936    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d61102-30e2-4861-a087-65b61421657b-kube-api-access-9rwqw" (OuterVolumeSpecName: "kube-api-access-9rwqw") pod "a4d61102-30e2-4861-a087-65b61421657b" (UID: "a4d61102-30e2-4861-a087-65b61421657b"). InnerVolumeSpecName "kube-api-access-9rwqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 12:48:41.454547 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.454506    2549 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4d61102-30e2-4861-a087-65b61421657b-webhook-cert\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:48:41.454547 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.454542    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rwqw\" (UniqueName: \"kubernetes.io/projected/a4d61102-30e2-4861-a087-65b61421657b-kube-api-access-9rwqw\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 12:48:41.633249 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.633215    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-jjl5g"]
Jul 28 12:48:41.636018 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:41.635988    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-jjl5g"]
Jul 28 12:48:42.518983 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.518945    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-vnqz4"]
Jul 28 12:48:42.519528 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.519509    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d61102-30e2-4861-a087-65b61421657b" containerName="manager"
Jul 28 12:48:42.519576 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.519531    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d61102-30e2-4861-a087-65b61421657b" containerName="manager"
Jul 28 12:48:42.519615 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.519598    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d61102-30e2-4861-a087-65b61421657b" containerName="manager"
Jul 28 12:48:42.523872 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.523857    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:48:42.528584 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.528558    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 12:48:42.528731 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.528558    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 12:48:42.528731 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.528570    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9jp9d\""
Jul 28 12:48:42.537248 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.537225    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-vnqz4"]
Jul 28 12:48:42.664939 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.664904    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a55f2b36-a833-46cf-8c4a-de402175e5a1-webhook-cert\") pod \"maas-controller-65dd8bc54b-vnqz4\" (UID: \"a55f2b36-a833-46cf-8c4a-de402175e5a1\") " pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:48:42.665105 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.664948    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v24vq\" (UniqueName: \"kubernetes.io/projected/a55f2b36-a833-46cf-8c4a-de402175e5a1-kube-api-access-v24vq\") pod \"maas-controller-65dd8bc54b-vnqz4\" (UID: \"a55f2b36-a833-46cf-8c4a-de402175e5a1\") " pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:48:42.692486 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.692442    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4d61102-30e2-4861-a087-65b61421657b" path="/var/lib/kubelet/pods/a4d61102-30e2-4861-a087-65b61421657b/volumes"
Jul 28 12:48:42.765939 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.765898    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a55f2b36-a833-46cf-8c4a-de402175e5a1-webhook-cert\") pod \"maas-controller-65dd8bc54b-vnqz4\" (UID: \"a55f2b36-a833-46cf-8c4a-de402175e5a1\") " pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:48:42.766130 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.765947    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v24vq\" (UniqueName: \"kubernetes.io/projected/a55f2b36-a833-46cf-8c4a-de402175e5a1-kube-api-access-v24vq\") pod \"maas-controller-65dd8bc54b-vnqz4\" (UID: \"a55f2b36-a833-46cf-8c4a-de402175e5a1\") " pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:48:42.768448 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.768428    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a55f2b36-a833-46cf-8c4a-de402175e5a1-webhook-cert\") pod \"maas-controller-65dd8bc54b-vnqz4\" (UID: \"a55f2b36-a833-46cf-8c4a-de402175e5a1\") " pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:48:42.776301 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.776245    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v24vq\" (UniqueName: \"kubernetes.io/projected/a55f2b36-a833-46cf-8c4a-de402175e5a1-kube-api-access-v24vq\") pod \"maas-controller-65dd8bc54b-vnqz4\" (UID: \"a55f2b36-a833-46cf-8c4a-de402175e5a1\") " pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:48:42.834398 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.834363    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:48:42.969692 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:42.969668    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65dd8bc54b-vnqz4"]
Jul 28 12:48:42.971388 ip-10-0-132-90 kubenswrapper[2549]: W0728 12:48:42.971360    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda55f2b36_a833_46cf_8c4a_de402175e5a1.slice/crio-5b8ae1d5645429d55f0e4667c3fdbb757dd1db0aa4baf9974bb24d2bb75494ee WatchSource:0}: Error finding container 5b8ae1d5645429d55f0e4667c3fdbb757dd1db0aa4baf9974bb24d2bb75494ee: Status 404 returned error can't find the container with id 5b8ae1d5645429d55f0e4667c3fdbb757dd1db0aa4baf9974bb24d2bb75494ee
Jul 28 12:48:43.320450 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:43.320407    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-vnqz4" event={"ID":"a55f2b36-a833-46cf-8c4a-de402175e5a1","Type":"ContainerStarted","Data":"5b8ae1d5645429d55f0e4667c3fdbb757dd1db0aa4baf9974bb24d2bb75494ee"}
Jul 28 12:48:44.327677 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:44.327619    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65dd8bc54b-vnqz4" event={"ID":"a55f2b36-a833-46cf-8c4a-de402175e5a1","Type":"ContainerStarted","Data":"a2be990dbb9379b456438a3e42e02f9050b1f8f55213bc70470207aeb3c357f7"}
Jul 28 12:48:44.328125 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:44.327717    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:48:44.348256 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:44.348207    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65dd8bc54b-vnqz4" podStartSLOduration=1.882189709 podStartE2EDuration="2.348193351s" totalImagesPullingTime="466.003642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:48:44.346041774 +0000 UTC m=+1116.210291636" watchObservedRunningTime="2026-07-28 12:48:44.348193351 +0000 UTC m=+1116.212443209"
Jul 28 12:48:55.335830 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:48:55.335797    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65dd8bc54b-vnqz4"
Jul 28 12:50:08.613038 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:50:08.613009    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:50:08.615272 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:50:08.615252    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:54:50.058275 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.058234    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"]
Jul 28 12:54:50.061590 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.061574    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:54:50.064508 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.064485    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5k6kz\""
Jul 28 12:54:50.078067 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.078046    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"]
Jul 28 12:54:50.194829 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.194793    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpw6w\" (UniqueName: \"kubernetes.io/projected/9899b67c-74bd-41e5-bd14-570f79d2204f-kube-api-access-tpw6w\") pod \"kuadrant-operator-controller-manager-86878c769f-hpn2c\" (UID: \"9899b67c-74bd-41e5-bd14-570f79d2204f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:54:50.195003 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.194853    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9899b67c-74bd-41e5-bd14-570f79d2204f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hpn2c\" (UID: \"9899b67c-74bd-41e5-bd14-570f79d2204f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:54:50.296019 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.295975    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9899b67c-74bd-41e5-bd14-570f79d2204f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hpn2c\" (UID: \"9899b67c-74bd-41e5-bd14-570f79d2204f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:54:50.296193 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.296065    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpw6w\" (UniqueName: \"kubernetes.io/projected/9899b67c-74bd-41e5-bd14-570f79d2204f-kube-api-access-tpw6w\") pod \"kuadrant-operator-controller-manager-86878c769f-hpn2c\" (UID: \"9899b67c-74bd-41e5-bd14-570f79d2204f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:54:50.296396 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.296375    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9899b67c-74bd-41e5-bd14-570f79d2204f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hpn2c\" (UID: \"9899b67c-74bd-41e5-bd14-570f79d2204f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:54:50.305491 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.305460    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpw6w\" (UniqueName: \"kubernetes.io/projected/9899b67c-74bd-41e5-bd14-570f79d2204f-kube-api-access-tpw6w\") pod \"kuadrant-operator-controller-manager-86878c769f-hpn2c\" (UID: \"9899b67c-74bd-41e5-bd14-570f79d2204f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:54:50.371884 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.371798    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:54:50.519340 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:50.519312    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"]
Jul 28 12:54:51.141793 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:51.141756    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c" event={"ID":"9899b67c-74bd-41e5-bd14-570f79d2204f","Type":"ContainerStarted","Data":"198d21402000e164a67f89154e7957bd3450ca316c643ecedae90bc4acffd813"}
Jul 28 12:54:51.142237 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:51.141833    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c" event={"ID":"9899b67c-74bd-41e5-bd14-570f79d2204f","Type":"ContainerStarted","Data":"a8d8f47999667cfc993701ff32a3f73e6a02b33063b4a940bea4c1c0e5583590"}
Jul 28 12:54:51.142237 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:51.141879    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:54:51.171522 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:54:51.171468    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c" podStartSLOduration=1.1714522729999999 podStartE2EDuration="1.171452273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:54:51.16915484 +0000 UTC m=+1483.033404700" watchObservedRunningTime="2026-07-28 12:54:51.171452273 +0000 UTC m=+1483.035702132"
Jul 28 12:55:02.150460 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:55:02.150428    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hpn2c"
Jul 28 12:55:08.644478 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:55:08.644363    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 12:55:08.647688 ip-10-0-132-90 kubenswrapper[2549]: I0728 12:55:08.646878    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:00:08.676320 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:00:08.676205    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:00:08.691876 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:00:08.691853    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:02:18.884608 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:18.884573    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"]
Jul 28 13:02:18.887383 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:18.887362    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:18.890470 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:18.890448    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d3416997-serving-cert\""
Jul 28 13:02:18.902045 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:18.902021    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"]
Jul 28 13:02:18.971393 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:18.971358    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-maas-api-tls\") pod \"maas-api-e2e-ait-d3416997-6d549b955-57lzc\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:18.971573 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:18.971433    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-openshift-service-ca\") pod \"maas-api-e2e-ait-d3416997-6d549b955-57lzc\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:18.971666 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:18.971571    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x47w\" (UniqueName: \"kubernetes.io/projected/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-kube-api-access-2x47w\") pod \"maas-api-e2e-ait-d3416997-6d549b955-57lzc\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:19.072125 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.072092    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x47w\" (UniqueName: \"kubernetes.io/projected/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-kube-api-access-2x47w\") pod \"maas-api-e2e-ait-d3416997-6d549b955-57lzc\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:19.072323 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.072185    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-maas-api-tls\") pod \"maas-api-e2e-ait-d3416997-6d549b955-57lzc\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:19.072483 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.072462    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-openshift-service-ca\") pod \"maas-api-e2e-ait-d3416997-6d549b955-57lzc\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:19.073322 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.073292    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-openshift-service-ca\") pod \"maas-api-e2e-ait-d3416997-6d549b955-57lzc\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:19.075508 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.075484    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-maas-api-tls\") pod \"maas-api-e2e-ait-d3416997-6d549b955-57lzc\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:19.083490 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.083471    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x47w\" (UniqueName: \"kubernetes.io/projected/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-kube-api-access-2x47w\") pod \"maas-api-e2e-ait-d3416997-6d549b955-57lzc\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:19.198389 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.198295    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:19.343334 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.343311    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"]
Jul 28 13:02:19.345745 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:02:19.345715    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b9c4bf_8ced_4440_bc5b_8a5acff9eeb5.slice/crio-93b2176dac0b2ba0527d886e91c3baa0a21d68c44bfe27cde022fdc4fa32f3e8 WatchSource:0}: Error finding container 93b2176dac0b2ba0527d886e91c3baa0a21d68c44bfe27cde022fdc4fa32f3e8: Status 404 returned error can't find the container with id 93b2176dac0b2ba0527d886e91c3baa0a21d68c44bfe27cde022fdc4fa32f3e8
Jul 28 13:02:19.347488 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.347463    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:02:19.442947 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:19.442909    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc" event={"ID":"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5","Type":"ContainerStarted","Data":"93b2176dac0b2ba0527d886e91c3baa0a21d68c44bfe27cde022fdc4fa32f3e8"}
Jul 28 13:02:20.449443 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:20.449406    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc" event={"ID":"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5","Type":"ContainerStarted","Data":"54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f"}
Jul 28 13:02:20.449918 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:20.449598    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:20.472068 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:20.472015    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc" podStartSLOduration=1.920582285 podStartE2EDuration="2.47199933s" totalImagesPullingTime="551.417045ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:02:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:02:20.47038029 +0000 UTC m=+1932.334630148" watchObservedRunningTime="2026-07-28 13:02:20.47199933 +0000 UTC m=+1932.336249188"
Jul 28 13:02:26.459439 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:26.459411    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:54.667881 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:54.667839    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"]
Jul 28 13:02:54.668755 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:54.668720    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc" podUID="d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5" containerName="maas-api" containerID="cri-o://54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f" gracePeriod=30
Jul 28 13:02:54.924308 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:54.924253    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:55.011684 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.011653    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-openshift-service-ca\") pod \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") "
Jul 28 13:02:55.011900 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.011704    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x47w\" (UniqueName: \"kubernetes.io/projected/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-kube-api-access-2x47w\") pod \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") "
Jul 28 13:02:55.011900 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.011751    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-maas-api-tls\") pod \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\" (UID: \"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5\") "
Jul 28 13:02:55.012119 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.012094    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5" (UID: "d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:02:55.014115 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.014088    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5" (UID: "d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:02:55.014220 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.014136    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-kube-api-access-2x47w" (OuterVolumeSpecName: "kube-api-access-2x47w") pod "d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5" (UID: "d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5"). InnerVolumeSpecName "kube-api-access-2x47w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:02:55.112964 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.112930    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x47w\" (UniqueName: \"kubernetes.io/projected/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-kube-api-access-2x47w\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:02:55.112964 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.112959    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:02:55.112964 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.112968    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:02:55.624226 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.624185    2549 generic.go:356] "Generic (PLEG): container finished" podID="d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5" containerID="54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f" exitCode=0
Jul 28 13:02:55.624226 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.624225    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"
Jul 28 13:02:55.624452 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.624253    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc" event={"ID":"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5","Type":"ContainerDied","Data":"54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f"}
Jul 28 13:02:55.624452 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.624305    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc" event={"ID":"d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5","Type":"ContainerDied","Data":"93b2176dac0b2ba0527d886e91c3baa0a21d68c44bfe27cde022fdc4fa32f3e8"}
Jul 28 13:02:55.624452 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.624340    2549 scope.go:117] "RemoveContainer" containerID="54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f"
Jul 28 13:02:55.633998 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.633976    2549 scope.go:117] "RemoveContainer" containerID="54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f"
Jul 28 13:02:55.634243 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:02:55.634224    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f\": container with ID starting with 54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f not found: ID does not exist" containerID="54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f"
Jul 28 13:02:55.634297 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.634251    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f"} err="failed to get container status \"54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f\": rpc error: code = NotFound desc = could not find container \"54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f\": container with ID starting with 54cd4ce5ca7ed27630181e4f7b758ecd4c55ed39c3331c67fff386d9b8208b9f not found: ID does not exist"
Jul 28 13:02:55.647107 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.647085    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"]
Jul 28 13:02:55.650920 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:55.650900    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3416997-6d549b955-57lzc"]
Jul 28 13:02:56.691974 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:02:56.691936    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5" path="/var/lib/kubelet/pods/d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5/volumes"
Jul 28 13:03:03.145160 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.145074    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"]
Jul 28 13:03:03.145575 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.145552    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5" containerName="maas-api"
Jul 28 13:03:03.145575 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.145566    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5" containerName="maas-api"
Jul 28 13:03:03.145722 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.145699    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0b9c4bf-8ced-4440-bc5b-8a5acff9eeb5" containerName="maas-api"
Jul 28 13:03:03.150173 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.150146    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.153316 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.153292    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-853a1b3f-serving-cert\""
Jul 28 13:03:03.162135 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.162111    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"]
Jul 28 13:03:03.289916 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.289875    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9zf8\" (UniqueName: \"kubernetes.io/projected/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-kube-api-access-v9zf8\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.290085 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.289942    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-openshift-service-ca\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.290085 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.289982    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-maas-api-tls\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.391195 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.391152    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9zf8\" (UniqueName: \"kubernetes.io/projected/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-kube-api-access-v9zf8\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.391444 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.391232    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-openshift-service-ca\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.391444 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.391262    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-maas-api-tls\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.391444 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:03:03.391384    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-serving-cert: secret "maas-api-e2e-migrate-853a1b3f-serving-cert" not found
Jul 28 13:03:03.391684 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:03:03.391466    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-maas-api-tls podName:4a9842c9-ffa3-4a38-97ed-7d512cadcc19 nodeName:}" failed. No retries permitted until 2026-07-28 13:03:03.891432055 +0000 UTC m=+1975.755681898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-maas-api-tls") pod "maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4" (UID: "4a9842c9-ffa3-4a38-97ed-7d512cadcc19") : secret "maas-api-e2e-migrate-853a1b3f-serving-cert" not found
Jul 28 13:03:03.392033 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.392008    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-openshift-service-ca\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.400823 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.400754    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9zf8\" (UniqueName: \"kubernetes.io/projected/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-kube-api-access-v9zf8\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.897592 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.897558    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-maas-api-tls\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:03.900402 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:03.900377    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-maas-api-tls\") pod \"maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:04.063647 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:04.063590    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:04.213867 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:04.213833    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"]
Jul 28 13:03:04.216694 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:03:04.216666    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9842c9_ffa3_4a38_97ed_7d512cadcc19.slice/crio-ee7affec1334aef5a37ebaade51f34b48331027b55a5519572b61338702a02cd WatchSource:0}: Error finding container ee7affec1334aef5a37ebaade51f34b48331027b55a5519572b61338702a02cd: Status 404 returned error can't find the container with id ee7affec1334aef5a37ebaade51f34b48331027b55a5519572b61338702a02cd
Jul 28 13:03:04.677679 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:04.677646    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4" event={"ID":"4a9842c9-ffa3-4a38-97ed-7d512cadcc19","Type":"ContainerStarted","Data":"ee7affec1334aef5a37ebaade51f34b48331027b55a5519572b61338702a02cd"}
Jul 28 13:03:06.693325 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:06.693300    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:06.693745 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:06.693341    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4" event={"ID":"4a9842c9-ffa3-4a38-97ed-7d512cadcc19","Type":"ContainerStarted","Data":"ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6"}
Jul 28 13:03:06.709617 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:06.709567    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4" podStartSLOduration=1.425936224 podStartE2EDuration="3.709550137s" totalImagesPullingTime="2.283613913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:03:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:03:06.708095986 +0000 UTC m=+1978.572345842" watchObservedRunningTime="2026-07-28 13:03:06.709550137 +0000 UTC m=+1978.573799996"
Jul 28 13:03:12.700848 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:12.700817    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:38.534275 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.534243    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"]
Jul 28 13:03:38.534882 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.534499    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4" podUID="4a9842c9-ffa3-4a38-97ed-7d512cadcc19" containerName="maas-api" containerID="cri-o://ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6" gracePeriod=30
Jul 28 13:03:38.802458 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.802433    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:38.849534 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.849505    2549 generic.go:356] "Generic (PLEG): container finished" podID="4a9842c9-ffa3-4a38-97ed-7d512cadcc19" containerID="ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6" exitCode=0
Jul 28 13:03:38.849720 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.849548    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"
Jul 28 13:03:38.849720 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.849591    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4" event={"ID":"4a9842c9-ffa3-4a38-97ed-7d512cadcc19","Type":"ContainerDied","Data":"ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6"}
Jul 28 13:03:38.849720 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.849665    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4" event={"ID":"4a9842c9-ffa3-4a38-97ed-7d512cadcc19","Type":"ContainerDied","Data":"ee7affec1334aef5a37ebaade51f34b48331027b55a5519572b61338702a02cd"}
Jul 28 13:03:38.849720 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.849696    2549 scope.go:117] "RemoveContainer" containerID="ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6"
Jul 28 13:03:38.858074 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.858053    2549 scope.go:117] "RemoveContainer" containerID="ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6"
Jul 28 13:03:38.858333 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:03:38.858314    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6\": container with ID starting with ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6 not found: ID does not exist" containerID="ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6"
Jul 28 13:03:38.858405 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.858346    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6"} err="failed to get container status \"ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6\": rpc error: code = NotFound desc = could not find container \"ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6\": container with ID starting with ef614e97352cdbd9aea29833470d9ec01f047e8ab6530ddd4d6a05a97b33d2b6 not found: ID does not exist"
Jul 28 13:03:38.910617 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.910583    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9zf8\" (UniqueName: \"kubernetes.io/projected/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-kube-api-access-v9zf8\") pod \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") "
Jul 28 13:03:38.910788 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.910669    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-openshift-service-ca\") pod \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") "
Jul 28 13:03:38.910788 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.910749    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-maas-api-tls\") pod \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\" (UID: \"4a9842c9-ffa3-4a38-97ed-7d512cadcc19\") "
Jul 28 13:03:38.911043 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.911016    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4a9842c9-ffa3-4a38-97ed-7d512cadcc19" (UID: "4a9842c9-ffa3-4a38-97ed-7d512cadcc19"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:03:38.912917 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.912891    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4a9842c9-ffa3-4a38-97ed-7d512cadcc19" (UID: "4a9842c9-ffa3-4a38-97ed-7d512cadcc19"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:03:38.912917 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:38.912905    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-kube-api-access-v9zf8" (OuterVolumeSpecName: "kube-api-access-v9zf8") pod "4a9842c9-ffa3-4a38-97ed-7d512cadcc19" (UID: "4a9842c9-ffa3-4a38-97ed-7d512cadcc19"). InnerVolumeSpecName "kube-api-access-v9zf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:03:39.011459 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:39.011424    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:03:39.011459 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:39.011453    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9zf8\" (UniqueName: \"kubernetes.io/projected/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-kube-api-access-v9zf8\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:03:39.011459 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:39.011467    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a9842c9-ffa3-4a38-97ed-7d512cadcc19-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:03:39.173276 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:39.173245    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"]
Jul 28 13:03:39.175270 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:39.175247    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-853a1b3f-56ddf99bdb-mfkz4"]
Jul 28 13:03:40.691855 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:40.691818    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a9842c9-ffa3-4a38-97ed-7d512cadcc19" path="/var/lib/kubelet/pods/4a9842c9-ffa3-4a38-97ed-7d512cadcc19/volumes"
Jul 28 13:03:46.300700 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.300663    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"]
Jul 28 13:03:46.301334 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.301312    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9842c9-ffa3-4a38-97ed-7d512cadcc19" containerName="maas-api"
Jul 28 13:03:46.301379 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.301340    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9842c9-ffa3-4a38-97ed-7d512cadcc19" containerName="maas-api"
Jul 28 13:03:46.301446 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.301433    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9842c9-ffa3-4a38-97ed-7d512cadcc19" containerName="maas-api"
Jul 28 13:03:46.304970 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.304945    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.308260 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.308233    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ce27c48c-serving-cert\""
Jul 28 13:03:46.319577 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.319553    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"]
Jul 28 13:03:46.478101 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.478061    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmbp8\" (UniqueName: \"kubernetes.io/projected/c755c185-9978-44e0-9945-abcba778a14b-kube-api-access-hmbp8\") pod \"maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.478308 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.478108    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c755c185-9978-44e0-9945-abcba778a14b-maas-api-tls\") pod \"maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.478308 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.478175    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c755c185-9978-44e0-9945-abcba778a14b-openshift-service-ca\") pod \"maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.579618 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.579580    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmbp8\" (UniqueName: \"kubernetes.io/projected/c755c185-9978-44e0-9945-abcba778a14b-kube-api-access-hmbp8\") pod \"maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.579837 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.579654    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c755c185-9978-44e0-9945-abcba778a14b-maas-api-tls\") pod \"maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.579837 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.579683    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c755c185-9978-44e0-9945-abcba778a14b-openshift-service-ca\") pod \"maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.580379 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.580352    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c755c185-9978-44e0-9945-abcba778a14b-openshift-service-ca\") pod \"maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.582320 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.582299    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c755c185-9978-44e0-9945-abcba778a14b-maas-api-tls\") pod \"maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.588687 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.588663    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmbp8\" (UniqueName: \"kubernetes.io/projected/c755c185-9978-44e0-9945-abcba778a14b-kube-api-access-hmbp8\") pod \"maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.616354 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.616318    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:46.770513 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.770482    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"]
Jul 28 13:03:46.773278 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:03:46.773247    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc755c185_9978_44e0_9945_abcba778a14b.slice/crio-dfd1be4987de19837a80550b05984495e7f580f7ef7adb5f924214fa455b4bd5 WatchSource:0}: Error finding container dfd1be4987de19837a80550b05984495e7f580f7ef7adb5f924214fa455b4bd5: Status 404 returned error can't find the container with id dfd1be4987de19837a80550b05984495e7f580f7ef7adb5f924214fa455b4bd5
Jul 28 13:03:46.892706 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:46.892612    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b" event={"ID":"c755c185-9978-44e0-9945-abcba778a14b","Type":"ContainerStarted","Data":"dfd1be4987de19837a80550b05984495e7f580f7ef7adb5f924214fa455b4bd5"}
Jul 28 13:03:47.899590 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:47.899546    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b" event={"ID":"c755c185-9978-44e0-9945-abcba778a14b","Type":"ContainerStarted","Data":"09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392"}
Jul 28 13:03:47.900088 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:47.899680    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:03:47.921190 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:47.921136    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b" podStartSLOduration=1.456284289 podStartE2EDuration="1.921121001s" totalImagesPullingTime="464.836712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:03:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:03:47.91947482 +0000 UTC m=+2019.783724679" watchObservedRunningTime="2026-07-28 13:03:47.921121001 +0000 UTC m=+2019.785370859"
Jul 28 13:03:53.910126 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:03:53.910095    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:04:22.545258 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.545221    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"]
Jul 28 13:04:22.545749 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.545489    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b" podUID="c755c185-9978-44e0-9945-abcba778a14b" containerName="maas-api" containerID="cri-o://09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392" gracePeriod=30
Jul 28 13:04:22.808180 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.808159    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:04:22.900514 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.900482    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c755c185-9978-44e0-9945-abcba778a14b-openshift-service-ca\") pod \"c755c185-9978-44e0-9945-abcba778a14b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") "
Jul 28 13:04:22.900732 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.900545    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmbp8\" (UniqueName: \"kubernetes.io/projected/c755c185-9978-44e0-9945-abcba778a14b-kube-api-access-hmbp8\") pod \"c755c185-9978-44e0-9945-abcba778a14b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") "
Jul 28 13:04:22.900732 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.900581    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c755c185-9978-44e0-9945-abcba778a14b-maas-api-tls\") pod \"c755c185-9978-44e0-9945-abcba778a14b\" (UID: \"c755c185-9978-44e0-9945-abcba778a14b\") "
Jul 28 13:04:22.900852 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.900825    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c755c185-9978-44e0-9945-abcba778a14b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c755c185-9978-44e0-9945-abcba778a14b" (UID: "c755c185-9978-44e0-9945-abcba778a14b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:04:22.900964 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.900944    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c755c185-9978-44e0-9945-abcba778a14b-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:04:22.902710 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.902687    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c755c185-9978-44e0-9945-abcba778a14b-kube-api-access-hmbp8" (OuterVolumeSpecName: "kube-api-access-hmbp8") pod "c755c185-9978-44e0-9945-abcba778a14b" (UID: "c755c185-9978-44e0-9945-abcba778a14b"). InnerVolumeSpecName "kube-api-access-hmbp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:04:22.902823 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:22.902761    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c755c185-9978-44e0-9945-abcba778a14b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c755c185-9978-44e0-9945-abcba778a14b" (UID: "c755c185-9978-44e0-9945-abcba778a14b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:04:23.001744 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.001708    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmbp8\" (UniqueName: \"kubernetes.io/projected/c755c185-9978-44e0-9945-abcba778a14b-kube-api-access-hmbp8\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:04:23.001744 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.001744    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c755c185-9978-44e0-9945-abcba778a14b-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:04:23.087989 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.087903    2549 generic.go:356] "Generic (PLEG): container finished" podID="c755c185-9978-44e0-9945-abcba778a14b" containerID="09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392" exitCode=0
Jul 28 13:04:23.087989 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.087944    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b" event={"ID":"c755c185-9978-44e0-9945-abcba778a14b","Type":"ContainerDied","Data":"09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392"}
Jul 28 13:04:23.087989 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.087950    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"
Jul 28 13:04:23.088237 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.087991    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b" event={"ID":"c755c185-9978-44e0-9945-abcba778a14b","Type":"ContainerDied","Data":"dfd1be4987de19837a80550b05984495e7f580f7ef7adb5f924214fa455b4bd5"}
Jul 28 13:04:23.088237 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.088019    2549 scope.go:117] "RemoveContainer" containerID="09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392"
Jul 28 13:04:23.098423 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.098405    2549 scope.go:117] "RemoveContainer" containerID="09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392"
Jul 28 13:04:23.098695 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:04:23.098673    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392\": container with ID starting with 09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392 not found: ID does not exist" containerID="09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392"
Jul 28 13:04:23.098780 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.098703    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392"} err="failed to get container status \"09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392\": rpc error: code = NotFound desc = could not find container \"09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392\": container with ID starting with 09c0e4cb404f830c4c7dedaeea91489265a04e387fbe504cd2217e8196a13392 not found: ID does not exist"
Jul 28 13:04:23.109940 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.109916    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"]
Jul 28 13:04:23.114032 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:23.114012    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ce27c48c-58ffc85bbf-hkf6b"]
Jul 28 13:04:24.692094 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:24.692059    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c755c185-9978-44e0-9945-abcba778a14b" path="/var/lib/kubelet/pods/c755c185-9978-44e0-9945-abcba778a14b/volumes"
Jul 28 13:04:31.662209 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.662126    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"]
Jul 28 13:04:31.662611 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.662595    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c755c185-9978-44e0-9945-abcba778a14b" containerName="maas-api"
Jul 28 13:04:31.662679 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.662613    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="c755c185-9978-44e0-9945-abcba778a14b" containerName="maas-api"
Jul 28 13:04:31.662715 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.662710    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="c755c185-9978-44e0-9945-abcba778a14b" containerName="maas-api"
Jul 28 13:04:31.666027 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.666009    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:31.669037 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.669014    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-bd87d0c9-serving-cert\""
Jul 28 13:04:31.675768 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.675733    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"]
Jul 28 13:04:31.782078 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.782040    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpth9\" (UniqueName: \"kubernetes.io/projected/8717a216-337d-4a82-bc84-7c940a6ba5f6-kube-api-access-mpth9\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:31.782255 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.782142    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8717a216-337d-4a82-bc84-7c940a6ba5f6-openshift-service-ca\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:31.782303 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.782261    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8717a216-337d-4a82-bc84-7c940a6ba5f6-maas-api-tls\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:31.882917 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.882883    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8717a216-337d-4a82-bc84-7c940a6ba5f6-maas-api-tls\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:31.883123 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.882943    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpth9\" (UniqueName: \"kubernetes.io/projected/8717a216-337d-4a82-bc84-7c940a6ba5f6-kube-api-access-mpth9\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:31.883123 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.882987    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8717a216-337d-4a82-bc84-7c940a6ba5f6-openshift-service-ca\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:31.883123 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:04:31.883037    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-serving-cert: secret "maas-api-e2e-derive-bd87d0c9-serving-cert" not found
Jul 28 13:04:31.883123 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:04:31.883116    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8717a216-337d-4a82-bc84-7c940a6ba5f6-maas-api-tls podName:8717a216-337d-4a82-bc84-7c940a6ba5f6 nodeName:}" failed. No retries permitted until 2026-07-28 13:04:32.383094626 +0000 UTC m=+2064.247344464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8717a216-337d-4a82-bc84-7c940a6ba5f6-maas-api-tls") pod "maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr" (UID: "8717a216-337d-4a82-bc84-7c940a6ba5f6") : secret "maas-api-e2e-derive-bd87d0c9-serving-cert" not found
Jul 28 13:04:31.883611 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.883592    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8717a216-337d-4a82-bc84-7c940a6ba5f6-openshift-service-ca\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:31.892475 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:31.892452    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpth9\" (UniqueName: \"kubernetes.io/projected/8717a216-337d-4a82-bc84-7c940a6ba5f6-kube-api-access-mpth9\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:32.387885 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:32.387846    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8717a216-337d-4a82-bc84-7c940a6ba5f6-maas-api-tls\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:32.390421 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:32.390401    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8717a216-337d-4a82-bc84-7c940a6ba5f6-maas-api-tls\") pod \"maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:32.578921 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:32.578886    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:32.723838 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:32.723808    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"]
Jul 28 13:04:32.727443 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:04:32.727310    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8717a216_337d_4a82_bc84_7c940a6ba5f6.slice/crio-620b91deeee58d84d4e92c118e8e3c0dbf3fc6b1699f80b88387e701bdde7dd5 WatchSource:0}: Error finding container 620b91deeee58d84d4e92c118e8e3c0dbf3fc6b1699f80b88387e701bdde7dd5: Status 404 returned error can't find the container with id 620b91deeee58d84d4e92c118e8e3c0dbf3fc6b1699f80b88387e701bdde7dd5
Jul 28 13:04:33.140541 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:33.140506    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr" event={"ID":"8717a216-337d-4a82-bc84-7c940a6ba5f6","Type":"ContainerStarted","Data":"620b91deeee58d84d4e92c118e8e3c0dbf3fc6b1699f80b88387e701bdde7dd5"}
Jul 28 13:04:34.149844 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:34.149803    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr" event={"ID":"8717a216-337d-4a82-bc84-7c940a6ba5f6","Type":"ContainerStarted","Data":"40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b"}
Jul 28 13:04:34.150327 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:34.149919    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:04:34.170848 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:34.170794    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr" podStartSLOduration=2.164406466 podStartE2EDuration="3.170772943s" totalImagesPullingTime="1.006366477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:04:34.168712618 +0000 UTC m=+2066.032962476" watchObservedRunningTime="2026-07-28 13:04:34.170772943 +0000 UTC m=+2066.035022803"
Jul 28 13:04:40.159472 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:04:40.159442    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:05:07.033930 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.033889    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"]
Jul 28 13:05:07.034527 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.034136    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr" podUID="8717a216-337d-4a82-bc84-7c940a6ba5f6" containerName="maas-api" containerID="cri-o://40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b" gracePeriod=30
Jul 28 13:05:07.309609 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.309587    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:05:07.324685 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.324648    2549 generic.go:356] "Generic (PLEG): container finished" podID="8717a216-337d-4a82-bc84-7c940a6ba5f6" containerID="40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b" exitCode=0
Jul 28 13:05:07.324685 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.324681    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"
Jul 28 13:05:07.324867 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.324684    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr" event={"ID":"8717a216-337d-4a82-bc84-7c940a6ba5f6","Type":"ContainerDied","Data":"40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b"}
Jul 28 13:05:07.324867 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.324733    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr" event={"ID":"8717a216-337d-4a82-bc84-7c940a6ba5f6","Type":"ContainerDied","Data":"620b91deeee58d84d4e92c118e8e3c0dbf3fc6b1699f80b88387e701bdde7dd5"}
Jul 28 13:05:07.324867 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.324755    2549 scope.go:117] "RemoveContainer" containerID="40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b"
Jul 28 13:05:07.334555 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.334522    2549 scope.go:117] "RemoveContainer" containerID="40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b"
Jul 28 13:05:07.334878 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:05:07.334858    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b\": container with ID starting with 40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b not found: ID does not exist" containerID="40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b"
Jul 28 13:05:07.334968 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.334891    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b"} err="failed to get container status \"40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b\": rpc error: code = NotFound desc = could not find container \"40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b\": container with ID starting with 40215284fdf5aa2b8adc218b67b03679dac55756dbeb3a9e2e8b88876edf200b not found: ID does not exist"
Jul 28 13:05:07.400412 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.400380    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpth9\" (UniqueName: \"kubernetes.io/projected/8717a216-337d-4a82-bc84-7c940a6ba5f6-kube-api-access-mpth9\") pod \"8717a216-337d-4a82-bc84-7c940a6ba5f6\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") "
Jul 28 13:05:07.400547 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.400424    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8717a216-337d-4a82-bc84-7c940a6ba5f6-openshift-service-ca\") pod \"8717a216-337d-4a82-bc84-7c940a6ba5f6\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") "
Jul 28 13:05:07.400547 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.400473    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8717a216-337d-4a82-bc84-7c940a6ba5f6-maas-api-tls\") pod \"8717a216-337d-4a82-bc84-7c940a6ba5f6\" (UID: \"8717a216-337d-4a82-bc84-7c940a6ba5f6\") "
Jul 28 13:05:07.400836 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.400815    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8717a216-337d-4a82-bc84-7c940a6ba5f6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8717a216-337d-4a82-bc84-7c940a6ba5f6" (UID: "8717a216-337d-4a82-bc84-7c940a6ba5f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:05:07.402770 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.402746    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8717a216-337d-4a82-bc84-7c940a6ba5f6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8717a216-337d-4a82-bc84-7c940a6ba5f6" (UID: "8717a216-337d-4a82-bc84-7c940a6ba5f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:05:07.402770 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.402762    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8717a216-337d-4a82-bc84-7c940a6ba5f6-kube-api-access-mpth9" (OuterVolumeSpecName: "kube-api-access-mpth9") pod "8717a216-337d-4a82-bc84-7c940a6ba5f6" (UID: "8717a216-337d-4a82-bc84-7c940a6ba5f6"). InnerVolumeSpecName "kube-api-access-mpth9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:05:07.501441 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.501400    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpth9\" (UniqueName: \"kubernetes.io/projected/8717a216-337d-4a82-bc84-7c940a6ba5f6-kube-api-access-mpth9\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:05:07.501441 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.501436    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8717a216-337d-4a82-bc84-7c940a6ba5f6-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:05:07.501441 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.501450    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8717a216-337d-4a82-bc84-7c940a6ba5f6-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:05:07.649938 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.649905    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"]
Jul 28 13:05:07.655635 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:07.655595    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bd87d0c9-69b986844b-7d7gr"]
Jul 28 13:05:08.695354 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:08.695241    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8717a216-337d-4a82-bc84-7c940a6ba5f6" path="/var/lib/kubelet/pods/8717a216-337d-4a82-bc84-7c940a6ba5f6/volumes"
Jul 28 13:05:08.714976 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:08.714955    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:05:08.723842 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:05:08.723823    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:07:07.330005 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.329974    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"]
Jul 28 13:07:07.330475 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.330396    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8717a216-337d-4a82-bc84-7c940a6ba5f6" containerName="maas-api"
Jul 28 13:07:07.330475 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.330409    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="8717a216-337d-4a82-bc84-7c940a6ba5f6" containerName="maas-api"
Jul 28 13:07:07.330564 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.330543    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="8717a216-337d-4a82-bc84-7c940a6ba5f6" containerName="maas-api"
Jul 28 13:07:07.333888 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.333869    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:07.337375 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.337357    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c499a1e4-serving-cert\""
Jul 28 13:07:07.343157 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.343136    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"]
Jul 28 13:07:07.482273 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.482234    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3874261-8419-4431-88b6-a44ab6f3413a-openshift-service-ca\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:07.482273 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.482275    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3874261-8419-4431-88b6-a44ab6f3413a-maas-api-tls\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:07.482479 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.482309    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fdlp\" (UniqueName: \"kubernetes.io/projected/a3874261-8419-4431-88b6-a44ab6f3413a-kube-api-access-7fdlp\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:07.583741 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.583650    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fdlp\" (UniqueName: \"kubernetes.io/projected/a3874261-8419-4431-88b6-a44ab6f3413a-kube-api-access-7fdlp\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:07.583888 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.583766    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3874261-8419-4431-88b6-a44ab6f3413a-openshift-service-ca\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:07.583888 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.583788    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3874261-8419-4431-88b6-a44ab6f3413a-maas-api-tls\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:07.583888 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:07:07.583879    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-serving-cert: secret "maas-api-e2e-mt-c499a1e4-serving-cert" not found
Jul 28 13:07:07.583996 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:07:07.583933    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3874261-8419-4431-88b6-a44ab6f3413a-maas-api-tls podName:a3874261-8419-4431-88b6-a44ab6f3413a nodeName:}" failed. No retries permitted until 2026-07-28 13:07:08.083915588 +0000 UTC m=+2219.948165426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a3874261-8419-4431-88b6-a44ab6f3413a-maas-api-tls") pod "maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g" (UID: "a3874261-8419-4431-88b6-a44ab6f3413a") : secret "maas-api-e2e-mt-c499a1e4-serving-cert" not found
Jul 28 13:07:07.584416 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.584396    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3874261-8419-4431-88b6-a44ab6f3413a-openshift-service-ca\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:07.601285 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:07.601251    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fdlp\" (UniqueName: \"kubernetes.io/projected/a3874261-8419-4431-88b6-a44ab6f3413a-kube-api-access-7fdlp\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:08.090400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:08.090360    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3874261-8419-4431-88b6-a44ab6f3413a-maas-api-tls\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:08.093049 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:08.093024    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3874261-8419-4431-88b6-a44ab6f3413a-maas-api-tls\") pod \"maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:08.244739 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:08.244686    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:08.402856 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:08.402831    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"]
Jul 28 13:07:08.405354 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:07:08.405330    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3874261_8419_4431_88b6_a44ab6f3413a.slice/crio-5cfbc8438b5b8e9ccb820ec00ce9f1b1f04d356151946e7b49b3682f8675513b WatchSource:0}: Error finding container 5cfbc8438b5b8e9ccb820ec00ce9f1b1f04d356151946e7b49b3682f8675513b: Status 404 returned error can't find the container with id 5cfbc8438b5b8e9ccb820ec00ce9f1b1f04d356151946e7b49b3682f8675513b
Jul 28 13:07:08.951482 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:08.951450    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g" event={"ID":"a3874261-8419-4431-88b6-a44ab6f3413a","Type":"ContainerStarted","Data":"5cfbc8438b5b8e9ccb820ec00ce9f1b1f04d356151946e7b49b3682f8675513b"}
Jul 28 13:07:09.959127 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:09.959089    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g" event={"ID":"a3874261-8419-4431-88b6-a44ab6f3413a","Type":"ContainerStarted","Data":"36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece"}
Jul 28 13:07:09.959523 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:09.959210    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:09.980848 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:09.980800    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g" podStartSLOduration=2.462943724 podStartE2EDuration="2.980785217s" totalImagesPullingTime="517.841493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:07:09.975964082 +0000 UTC m=+2221.840213942" watchObservedRunningTime="2026-07-28 13:07:09.980785217 +0000 UTC m=+2221.845035075"
Jul 28 13:07:15.970834 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:15.970803    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:29.676236 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:29.676148    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"]
Jul 28 13:07:29.676665 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:29.676401    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g" podUID="a3874261-8419-4431-88b6-a44ab6f3413a" containerName="maas-api" containerID="cri-o://36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece" gracePeriod=30
Jul 28 13:07:29.940993 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:29.940919    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:29.999331 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:29.999292    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fdlp\" (UniqueName: \"kubernetes.io/projected/a3874261-8419-4431-88b6-a44ab6f3413a-kube-api-access-7fdlp\") pod \"a3874261-8419-4431-88b6-a44ab6f3413a\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") "
Jul 28 13:07:29.999519 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:29.999388    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3874261-8419-4431-88b6-a44ab6f3413a-maas-api-tls\") pod \"a3874261-8419-4431-88b6-a44ab6f3413a\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") "
Jul 28 13:07:29.999519 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:29.999437    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3874261-8419-4431-88b6-a44ab6f3413a-openshift-service-ca\") pod \"a3874261-8419-4431-88b6-a44ab6f3413a\" (UID: \"a3874261-8419-4431-88b6-a44ab6f3413a\") "
Jul 28 13:07:29.999913 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:29.999877    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3874261-8419-4431-88b6-a44ab6f3413a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3874261-8419-4431-88b6-a44ab6f3413a" (UID: "a3874261-8419-4431-88b6-a44ab6f3413a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:07:30.001517 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.001485    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3874261-8419-4431-88b6-a44ab6f3413a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3874261-8419-4431-88b6-a44ab6f3413a" (UID: "a3874261-8419-4431-88b6-a44ab6f3413a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:07:30.001617 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.001525    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3874261-8419-4431-88b6-a44ab6f3413a-kube-api-access-7fdlp" (OuterVolumeSpecName: "kube-api-access-7fdlp") pod "a3874261-8419-4431-88b6-a44ab6f3413a" (UID: "a3874261-8419-4431-88b6-a44ab6f3413a"). InnerVolumeSpecName "kube-api-access-7fdlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:07:30.071605 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.071566    2549 generic.go:356] "Generic (PLEG): container finished" podID="a3874261-8419-4431-88b6-a44ab6f3413a" containerID="36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece" exitCode=0
Jul 28 13:07:30.071813 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.071611    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"
Jul 28 13:07:30.071813 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.071659    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g" event={"ID":"a3874261-8419-4431-88b6-a44ab6f3413a","Type":"ContainerDied","Data":"36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece"}
Jul 28 13:07:30.071813 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.071715    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g" event={"ID":"a3874261-8419-4431-88b6-a44ab6f3413a","Type":"ContainerDied","Data":"5cfbc8438b5b8e9ccb820ec00ce9f1b1f04d356151946e7b49b3682f8675513b"}
Jul 28 13:07:30.071813 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.071737    2549 scope.go:117] "RemoveContainer" containerID="36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece"
Jul 28 13:07:30.082741 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.082709    2549 scope.go:117] "RemoveContainer" containerID="36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece"
Jul 28 13:07:30.083170 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:07:30.083054    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece\": container with ID starting with 36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece not found: ID does not exist" containerID="36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece"
Jul 28 13:07:30.083170 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.083119    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece"} err="failed to get container status \"36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece\": rpc error: code = NotFound desc = could not find container \"36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece\": container with ID starting with 36774178bd8db2238c1a9dc0a9e0d82b97642546bc609a78af21c2d87df17ece not found: ID does not exist"
Jul 28 13:07:30.097250 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.097209    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"]
Jul 28 13:07:30.100972 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.100943    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fdlp\" (UniqueName: \"kubernetes.io/projected/a3874261-8419-4431-88b6-a44ab6f3413a-kube-api-access-7fdlp\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:07:30.100972 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.100969    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3874261-8419-4431-88b6-a44ab6f3413a-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:07:30.101165 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.100980    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3874261-8419-4431-88b6-a44ab6f3413a-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:07:30.101717 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.101697    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c499a1e4-6ffdcbc569-l682g"]
Jul 28 13:07:30.692873 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:07:30.692830    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3874261-8419-4431-88b6-a44ab6f3413a" path="/var/lib/kubelet/pods/a3874261-8419-4431-88b6-a44ab6f3413a/volumes"
Jul 28 13:08:04.529877 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.529844    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"]
Jul 28 13:08:04.530346 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.530276    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3874261-8419-4431-88b6-a44ab6f3413a" containerName="maas-api"
Jul 28 13:08:04.530346 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.530290    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3874261-8419-4431-88b6-a44ab6f3413a" containerName="maas-api"
Jul 28 13:08:04.530422 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.530388    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3874261-8419-4431-88b6-a44ab6f3413a" containerName="maas-api"
Jul 28 13:08:04.533732 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.533716    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:04.537126 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.537100    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0e7d2b19-serving-cert\""
Jul 28 13:08:04.549234 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.549205    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"]
Jul 28 13:08:04.610222 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.610189    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbde7b9a-61d1-432a-9910-4d60e3f24026-openshift-service-ca\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:04.610382 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.610234    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqlbb\" (UniqueName: \"kubernetes.io/projected/cbde7b9a-61d1-432a-9910-4d60e3f24026-kube-api-access-qqlbb\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:04.610382 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.610258    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbde7b9a-61d1-432a-9910-4d60e3f24026-maas-api-tls\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:04.710839 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.710795    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbde7b9a-61d1-432a-9910-4d60e3f24026-maas-api-tls\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:04.711018 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.710894    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbde7b9a-61d1-432a-9910-4d60e3f24026-openshift-service-ca\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:04.711018 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.710930    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqlbb\" (UniqueName: \"kubernetes.io/projected/cbde7b9a-61d1-432a-9910-4d60e3f24026-kube-api-access-qqlbb\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:04.711018 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:08:04.710934    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-serving-cert: secret "maas-api-e2e-mt-0e7d2b19-serving-cert" not found
Jul 28 13:08:04.711174 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:08:04.711026    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbde7b9a-61d1-432a-9910-4d60e3f24026-maas-api-tls podName:cbde7b9a-61d1-432a-9910-4d60e3f24026 nodeName:}" failed. No retries permitted until 2026-07-28 13:08:05.211008384 +0000 UTC m=+2277.075258225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cbde7b9a-61d1-432a-9910-4d60e3f24026-maas-api-tls") pod "maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq" (UID: "cbde7b9a-61d1-432a-9910-4d60e3f24026") : secret "maas-api-e2e-mt-0e7d2b19-serving-cert" not found
Jul 28 13:08:04.711479 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.711462    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbde7b9a-61d1-432a-9910-4d60e3f24026-openshift-service-ca\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:04.721519 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:04.721491    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqlbb\" (UniqueName: \"kubernetes.io/projected/cbde7b9a-61d1-432a-9910-4d60e3f24026-kube-api-access-qqlbb\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:05.215098 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:05.215064    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbde7b9a-61d1-432a-9910-4d60e3f24026-maas-api-tls\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:05.217555 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:05.217531    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbde7b9a-61d1-432a-9910-4d60e3f24026-maas-api-tls\") pod \"maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:05.444431 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:05.444393    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:05.585961 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:05.585932    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"]
Jul 28 13:08:05.587571 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:08:05.587545    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbde7b9a_61d1_432a_9910_4d60e3f24026.slice/crio-5b4af0c1cddba43d771f19cbc679afeb04b6d8f41d9656ddff38810729506a01 WatchSource:0}: Error finding container 5b4af0c1cddba43d771f19cbc679afeb04b6d8f41d9656ddff38810729506a01: Status 404 returned error can't find the container with id 5b4af0c1cddba43d771f19cbc679afeb04b6d8f41d9656ddff38810729506a01
Jul 28 13:08:05.588957 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:05.588937    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:08:06.272273 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:06.272201    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq" event={"ID":"cbde7b9a-61d1-432a-9910-4d60e3f24026","Type":"ContainerStarted","Data":"5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9"}
Jul 28 13:08:06.272463 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:06.272300    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq" event={"ID":"cbde7b9a-61d1-432a-9910-4d60e3f24026","Type":"ContainerStarted","Data":"5b4af0c1cddba43d771f19cbc679afeb04b6d8f41d9656ddff38810729506a01"}
Jul 28 13:08:06.272463 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:06.272350    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:06.296835 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:06.296779    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq" podStartSLOduration=1.811192749 podStartE2EDuration="2.296762461s" totalImagesPullingTime="485.569712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:08:06.295158804 +0000 UTC m=+2278.159408657" watchObservedRunningTime="2026-07-28 13:08:06.296762461 +0000 UTC m=+2278.161012454"
Jul 28 13:08:12.283893 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:12.283861    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:53.965031 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:53.964997    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"]
Jul 28 13:08:53.965975 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:53.965916    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq" podUID="cbde7b9a-61d1-432a-9910-4d60e3f24026" containerName="maas-api" containerID="cri-o://5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9" gracePeriod=30
Jul 28 13:08:54.236023 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.235999    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:54.349091 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.349059    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqlbb\" (UniqueName: \"kubernetes.io/projected/cbde7b9a-61d1-432a-9910-4d60e3f24026-kube-api-access-qqlbb\") pod \"cbde7b9a-61d1-432a-9910-4d60e3f24026\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") "
Jul 28 13:08:54.349277 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.349181    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbde7b9a-61d1-432a-9910-4d60e3f24026-openshift-service-ca\") pod \"cbde7b9a-61d1-432a-9910-4d60e3f24026\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") "
Jul 28 13:08:54.349277 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.349218    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbde7b9a-61d1-432a-9910-4d60e3f24026-maas-api-tls\") pod \"cbde7b9a-61d1-432a-9910-4d60e3f24026\" (UID: \"cbde7b9a-61d1-432a-9910-4d60e3f24026\") "
Jul 28 13:08:54.349739 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.349690    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbde7b9a-61d1-432a-9910-4d60e3f24026-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cbde7b9a-61d1-432a-9910-4d60e3f24026" (UID: "cbde7b9a-61d1-432a-9910-4d60e3f24026"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:08:54.351596 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.351560    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbde7b9a-61d1-432a-9910-4d60e3f24026-kube-api-access-qqlbb" (OuterVolumeSpecName: "kube-api-access-qqlbb") pod "cbde7b9a-61d1-432a-9910-4d60e3f24026" (UID: "cbde7b9a-61d1-432a-9910-4d60e3f24026"). InnerVolumeSpecName "kube-api-access-qqlbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:08:54.351842 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.351817    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbde7b9a-61d1-432a-9910-4d60e3f24026-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cbde7b9a-61d1-432a-9910-4d60e3f24026" (UID: "cbde7b9a-61d1-432a-9910-4d60e3f24026"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:08:54.450201 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.450169    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbde7b9a-61d1-432a-9910-4d60e3f24026-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:08:54.450201 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.450199    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqlbb\" (UniqueName: \"kubernetes.io/projected/cbde7b9a-61d1-432a-9910-4d60e3f24026-kube-api-access-qqlbb\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:08:54.450201 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.450211    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbde7b9a-61d1-432a-9910-4d60e3f24026-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:08:54.522462 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.522370    2549 generic.go:356] "Generic (PLEG): container finished" podID="cbde7b9a-61d1-432a-9910-4d60e3f24026" containerID="5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9" exitCode=0
Jul 28 13:08:54.522614 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.522462    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"
Jul 28 13:08:54.522614 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.522449    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq" event={"ID":"cbde7b9a-61d1-432a-9910-4d60e3f24026","Type":"ContainerDied","Data":"5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9"}
Jul 28 13:08:54.522614 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.522591    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq" event={"ID":"cbde7b9a-61d1-432a-9910-4d60e3f24026","Type":"ContainerDied","Data":"5b4af0c1cddba43d771f19cbc679afeb04b6d8f41d9656ddff38810729506a01"}
Jul 28 13:08:54.522807 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.522617    2549 scope.go:117] "RemoveContainer" containerID="5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9"
Jul 28 13:08:54.532878 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.532860    2549 scope.go:117] "RemoveContainer" containerID="5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9"
Jul 28 13:08:54.533127 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:08:54.533106    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9\": container with ID starting with 5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9 not found: ID does not exist" containerID="5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9"
Jul 28 13:08:54.533193 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.533140    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9"} err="failed to get container status \"5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9\": rpc error: code = NotFound desc = could not find container \"5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9\": container with ID starting with 5727e7233b9dd3eba2df391827ea920863f98c0c1a62521e84844db2f545dae9 not found: ID does not exist"
Jul 28 13:08:54.545353 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.545332    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"]
Jul 28 13:08:54.547267 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.547240    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e7d2b19-6b9bdcccdf-7pljq"]
Jul 28 13:08:54.692732 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:08:54.692698    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbde7b9a-61d1-432a-9910-4d60e3f24026" path="/var/lib/kubelet/pods/cbde7b9a-61d1-432a-9910-4d60e3f24026/volumes"
Jul 28 13:10:08.748339 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:08.748218    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:10:08.759605 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:08.759562    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:10:49.158673 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.158642    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"]
Jul 28 13:10:49.159145 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.159064    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbde7b9a-61d1-432a-9910-4d60e3f24026" containerName="maas-api"
Jul 28 13:10:49.159145 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.159078    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbde7b9a-61d1-432a-9910-4d60e3f24026" containerName="maas-api"
Jul 28 13:10:49.159218 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.159147    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbde7b9a-61d1-432a-9910-4d60e3f24026" containerName="maas-api"
Jul 28 13:10:49.162070 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.162049    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.165523 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.165490    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c260d4-serving-cert\""
Jul 28 13:10:49.177004 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.176981    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"]
Jul 28 13:10:49.217228 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.217200    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.217388 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.217254    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sn66\" (UniqueName: \"kubernetes.io/projected/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-kube-api-access-9sn66\") pod \"maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.217388 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.217301    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.318705 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.318664    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sn66\" (UniqueName: \"kubernetes.io/projected/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-kube-api-access-9sn66\") pod \"maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.318705 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.318710    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.318961 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.318791    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.319350 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.319326    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.321361 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.321341    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.328353 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.328327    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sn66\" (UniqueName: \"kubernetes.io/projected/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-kube-api-access-9sn66\") pod \"maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.472315 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.472219    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:49.624318 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:49.624292    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"]
Jul 28 13:10:49.625558 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:10:49.625526    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3cb9e4c_d936_4ab8_a2eb_714a54fbe9f4.slice/crio-f6078ab2f8a7ef6b34c15c22d6d02e434a0154f40c3a32c4090843b0f8ca4d93 WatchSource:0}: Error finding container f6078ab2f8a7ef6b34c15c22d6d02e434a0154f40c3a32c4090843b0f8ca4d93: Status 404 returned error can't find the container with id f6078ab2f8a7ef6b34c15c22d6d02e434a0154f40c3a32c4090843b0f8ca4d93
Jul 28 13:10:50.125567 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:50.125537    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm" event={"ID":"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4","Type":"ContainerStarted","Data":"56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded"}
Jul 28 13:10:50.125750 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:50.125586    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm" event={"ID":"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4","Type":"ContainerStarted","Data":"f6078ab2f8a7ef6b34c15c22d6d02e434a0154f40c3a32c4090843b0f8ca4d93"}
Jul 28 13:10:50.125750 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:50.125664    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:50.148436 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:50.148388    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm" podStartSLOduration=0.779758879 podStartE2EDuration="1.148374135s" totalImagesPullingTime="368.615256ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:10:50.147126405 +0000 UTC m=+2442.011376264" watchObservedRunningTime="2026-07-28 13:10:50.148374135 +0000 UTC m=+2442.012623993"
Jul 28 13:10:56.136947 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:56.136917    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:10:57.336841 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.336802    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"]
Jul 28 13:10:57.340764 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.340744    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.343758 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.343730    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e75cd7-serving-cert\""
Jul 28 13:10:57.351591 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.351571    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"]
Jul 28 13:10:57.390594 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.390558    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0151f1ec-56b3-40ee-93f7-23241eb89469-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.390766 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.390614    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0151f1ec-56b3-40ee-93f7-23241eb89469-maas-api-tls\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.390766 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.390711    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sjxf\" (UniqueName: \"kubernetes.io/projected/0151f1ec-56b3-40ee-93f7-23241eb89469-kube-api-access-8sjxf\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.492158 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.492118    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sjxf\" (UniqueName: \"kubernetes.io/projected/0151f1ec-56b3-40ee-93f7-23241eb89469-kube-api-access-8sjxf\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.492359 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.492171    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0151f1ec-56b3-40ee-93f7-23241eb89469-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.492359 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.492207    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0151f1ec-56b3-40ee-93f7-23241eb89469-maas-api-tls\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.492359 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:10:57.492342    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-serving-cert: secret "maas-api-e2e-inf-b-e75cd7-serving-cert" not found
Jul 28 13:10:57.492544 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:10:57.492421    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0151f1ec-56b3-40ee-93f7-23241eb89469-maas-api-tls podName:0151f1ec-56b3-40ee-93f7-23241eb89469 nodeName:}" failed. No retries permitted until 2026-07-28 13:10:57.992400562 +0000 UTC m=+2449.856650399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0151f1ec-56b3-40ee-93f7-23241eb89469-maas-api-tls") pod "maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs" (UID: "0151f1ec-56b3-40ee-93f7-23241eb89469") : secret "maas-api-e2e-inf-b-e75cd7-serving-cert" not found
Jul 28 13:10:57.492920 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.492899    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0151f1ec-56b3-40ee-93f7-23241eb89469-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.501419 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.501398    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sjxf\" (UniqueName: \"kubernetes.io/projected/0151f1ec-56b3-40ee-93f7-23241eb89469-kube-api-access-8sjxf\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.968572 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.968540    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"]
Jul 28 13:10:57.973153 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.973125    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:10:57.976459 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.976431    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e75cd7-dockercfg-x7vk8\""
Jul 28 13:10:57.976459 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.976431    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e75cd7\""
Jul 28 13:10:57.984065 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.984041    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"]
Jul 28 13:10:57.998034 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.998007    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spzf6\" (UniqueName: \"kubernetes.io/projected/590f1ae5-4537-433b-8849-1a4c13bf55cf-kube-api-access-spzf6\") pod \"payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl\" (UID: \"590f1ae5-4537-433b-8849-1a4c13bf55cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:10:57.998165 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.998063    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0151f1ec-56b3-40ee-93f7-23241eb89469-maas-api-tls\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:57.998165 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:57.998087    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/590f1ae5-4537-433b-8849-1a4c13bf55cf-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl\" (UID: \"590f1ae5-4537-433b-8849-1a4c13bf55cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:10:58.000553 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:58.000533    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0151f1ec-56b3-40ee-93f7-23241eb89469-maas-api-tls\") pod \"maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:58.099310 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:58.099268    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/590f1ae5-4537-433b-8849-1a4c13bf55cf-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl\" (UID: \"590f1ae5-4537-433b-8849-1a4c13bf55cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:10:58.099477 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:58.099385    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spzf6\" (UniqueName: \"kubernetes.io/projected/590f1ae5-4537-433b-8849-1a4c13bf55cf-kube-api-access-spzf6\") pod \"payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl\" (UID: \"590f1ae5-4537-433b-8849-1a4c13bf55cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:10:58.099969 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:58.099949    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/590f1ae5-4537-433b-8849-1a4c13bf55cf-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl\" (UID: \"590f1ae5-4537-433b-8849-1a4c13bf55cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:10:58.108342 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:58.108320    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spzf6\" (UniqueName: \"kubernetes.io/projected/590f1ae5-4537-433b-8849-1a4c13bf55cf-kube-api-access-spzf6\") pod \"payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl\" (UID: \"590f1ae5-4537-433b-8849-1a4c13bf55cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:10:58.253233 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:58.253129    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:58.287476 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:58.287447    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:10:58.417561 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:58.417517    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"]
Jul 28 13:10:58.420524 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:10:58.420496    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0151f1ec_56b3_40ee_93f7_23241eb89469.slice/crio-1d6b0e4157795791ca4e6f4e841de3a1ab35690f3e1b6910b692cf6bb0bb3826 WatchSource:0}: Error finding container 1d6b0e4157795791ca4e6f4e841de3a1ab35690f3e1b6910b692cf6bb0bb3826: Status 404 returned error can't find the container with id 1d6b0e4157795791ca4e6f4e841de3a1ab35690f3e1b6910b692cf6bb0bb3826
Jul 28 13:10:58.672585 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:58.672556    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"]
Jul 28 13:10:58.675233 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:10:58.675204    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590f1ae5_4537_433b_8849_1a4c13bf55cf.slice/crio-d4d8c54165175cead21a0045e5aa9df81f08506d860df8da41121ccea710e379 WatchSource:0}: Error finding container d4d8c54165175cead21a0045e5aa9df81f08506d860df8da41121ccea710e379: Status 404 returned error can't find the container with id d4d8c54165175cead21a0045e5aa9df81f08506d860df8da41121ccea710e379
Jul 28 13:10:59.192091 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:59.192052    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs" event={"ID":"0151f1ec-56b3-40ee-93f7-23241eb89469","Type":"ContainerStarted","Data":"306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b"}
Jul 28 13:10:59.192402 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:59.192372    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:10:59.192539 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:59.192431    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs" event={"ID":"0151f1ec-56b3-40ee-93f7-23241eb89469","Type":"ContainerStarted","Data":"1d6b0e4157795791ca4e6f4e841de3a1ab35690f3e1b6910b692cf6bb0bb3826"}
Jul 28 13:10:59.193816 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:59.193794    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl" event={"ID":"590f1ae5-4537-433b-8849-1a4c13bf55cf","Type":"ContainerStarted","Data":"baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4"}
Jul 28 13:10:59.193927 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:59.193831    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl" event={"ID":"590f1ae5-4537-433b-8849-1a4c13bf55cf","Type":"ContainerStarted","Data":"d4d8c54165175cead21a0045e5aa9df81f08506d860df8da41121ccea710e379"}
Jul 28 13:10:59.193927 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:59.193869    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:10:59.209262 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:59.209225    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs" podStartSLOduration=1.896667461 podStartE2EDuration="2.209205642s" totalImagesPullingTime="312.538181ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:10:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:10:59.209065513 +0000 UTC m=+2451.073315385" watchObservedRunningTime="2026-07-28 13:10:59.209205642 +0000 UTC m=+2451.073455504"
Jul 28 13:10:59.227376 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:10:59.227327    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl" podStartSLOduration=2.227312875 podStartE2EDuration="2.227312875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:10:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:10:59.225085383 +0000 UTC m=+2451.089335242" watchObservedRunningTime="2026-07-28 13:10:59.227312875 +0000 UTC m=+2451.091562756"
Jul 28 13:11:05.204217 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:11:05.204187    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:11:10.200951 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:11:10.200860    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:12:32.993199 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:32.993164    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"]
Jul 28 13:12:32.993845 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:32.993397    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm" podUID="d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4" containerName="maas-api" containerID="cri-o://56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded" gracePeriod=30
Jul 28 13:12:33.252987 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.252920    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:12:33.279206 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.279129    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-openshift-service-ca\") pod \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") "
Jul 28 13:12:33.279419 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.279266    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sn66\" (UniqueName: \"kubernetes.io/projected/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-kube-api-access-9sn66\") pod \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") "
Jul 28 13:12:33.279419 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.279369    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-maas-api-tls\") pod \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\" (UID: \"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4\") "
Jul 28 13:12:33.279551 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.279520    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4" (UID: "d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:12:33.279714 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.279696    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:12:33.282287 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.282255    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4" (UID: "d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:12:33.282287 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.282277    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-kube-api-access-9sn66" (OuterVolumeSpecName: "kube-api-access-9sn66") pod "d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4" (UID: "d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4"). InnerVolumeSpecName "kube-api-access-9sn66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:12:33.380331 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.380297    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:12:33.380331 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.380325    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sn66\" (UniqueName: \"kubernetes.io/projected/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4-kube-api-access-9sn66\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:12:33.679434 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.679393    2549 generic.go:356] "Generic (PLEG): container finished" podID="d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4" containerID="56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded" exitCode=0
Jul 28 13:12:33.679647 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.679439    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"
Jul 28 13:12:33.679647 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.679483    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm" event={"ID":"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4","Type":"ContainerDied","Data":"56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded"}
Jul 28 13:12:33.679647 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.679546    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm" event={"ID":"d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4","Type":"ContainerDied","Data":"f6078ab2f8a7ef6b34c15c22d6d02e434a0154f40c3a32c4090843b0f8ca4d93"}
Jul 28 13:12:33.679647 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.679574    2549 scope.go:117] "RemoveContainer" containerID="56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded"
Jul 28 13:12:33.689652 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.689615    2549 scope.go:117] "RemoveContainer" containerID="56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded"
Jul 28 13:12:33.689932 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:12:33.689913    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded\": container with ID starting with 56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded not found: ID does not exist" containerID="56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded"
Jul 28 13:12:33.690009 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.689939    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded"} err="failed to get container status \"56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded\": rpc error: code = NotFound desc = could not find container \"56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded\": container with ID starting with 56fe28ec5977dab7ab4894b5c79fca43b6cad24c390790a66a9b4435e8e6eded not found: ID does not exist"
Jul 28 13:12:33.702199 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.702167    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"]
Jul 28 13:12:33.710225 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:33.710201    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c260d4-c966fcc58-v5lkm"]
Jul 28 13:12:34.692444 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:34.692414    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4" path="/var/lib/kubelet/pods/d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4/volumes"
Jul 28 13:12:54.843581 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:54.843544    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"]
Jul 28 13:12:54.844114 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:54.843829    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs" podUID="0151f1ec-56b3-40ee-93f7-23241eb89469" containerName="maas-api" containerID="cri-o://306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b" gracePeriod=30
Jul 28 13:12:54.895975 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:54.895939    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"]
Jul 28 13:12:54.896228 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:54.896198    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl" podUID="590f1ae5-4537-433b-8849-1a4c13bf55cf" containerName="payload-processing" containerID="cri-o://baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4" gracePeriod=30
Jul 28 13:12:55.187116 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.187090    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:12:55.199972 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.199941    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs" podUID="0151f1ec-56b3-40ee-93f7-23241eb89469" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.57:8443/health\": dial tcp 10.133.0.57:8443: connect: connection refused"
Jul 28 13:12:55.277088 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.277058    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spzf6\" (UniqueName: \"kubernetes.io/projected/590f1ae5-4537-433b-8849-1a4c13bf55cf-kube-api-access-spzf6\") pod \"590f1ae5-4537-433b-8849-1a4c13bf55cf\" (UID: \"590f1ae5-4537-433b-8849-1a4c13bf55cf\") "
Jul 28 13:12:55.277266 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.277222    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/590f1ae5-4537-433b-8849-1a4c13bf55cf-plugins-config-volume\") pod \"590f1ae5-4537-433b-8849-1a4c13bf55cf\" (UID: \"590f1ae5-4537-433b-8849-1a4c13bf55cf\") "
Jul 28 13:12:55.277555 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.277530    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/590f1ae5-4537-433b-8849-1a4c13bf55cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "590f1ae5-4537-433b-8849-1a4c13bf55cf" (UID: "590f1ae5-4537-433b-8849-1a4c13bf55cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:12:55.279247 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.279216    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/590f1ae5-4537-433b-8849-1a4c13bf55cf-kube-api-access-spzf6" (OuterVolumeSpecName: "kube-api-access-spzf6") pod "590f1ae5-4537-433b-8849-1a4c13bf55cf" (UID: "590f1ae5-4537-433b-8849-1a4c13bf55cf"). InnerVolumeSpecName "kube-api-access-spzf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:12:55.310662 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.310642    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:12:55.378538 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.378457    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0151f1ec-56b3-40ee-93f7-23241eb89469-maas-api-tls\") pod \"0151f1ec-56b3-40ee-93f7-23241eb89469\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") "
Jul 28 13:12:55.378538 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.378509    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0151f1ec-56b3-40ee-93f7-23241eb89469-openshift-service-ca\") pod \"0151f1ec-56b3-40ee-93f7-23241eb89469\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") "
Jul 28 13:12:55.378759 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.378710    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sjxf\" (UniqueName: \"kubernetes.io/projected/0151f1ec-56b3-40ee-93f7-23241eb89469-kube-api-access-8sjxf\") pod \"0151f1ec-56b3-40ee-93f7-23241eb89469\" (UID: \"0151f1ec-56b3-40ee-93f7-23241eb89469\") "
Jul 28 13:12:55.378912 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.378877    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0151f1ec-56b3-40ee-93f7-23241eb89469-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0151f1ec-56b3-40ee-93f7-23241eb89469" (UID: "0151f1ec-56b3-40ee-93f7-23241eb89469"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:12:55.379124 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.379108    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0151f1ec-56b3-40ee-93f7-23241eb89469-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:12:55.379198 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.379133    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/590f1ae5-4537-433b-8849-1a4c13bf55cf-plugins-config-volume\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:12:55.379198 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.379148    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spzf6\" (UniqueName: \"kubernetes.io/projected/590f1ae5-4537-433b-8849-1a4c13bf55cf-kube-api-access-spzf6\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:12:55.380748 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.380716    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0151f1ec-56b3-40ee-93f7-23241eb89469-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0151f1ec-56b3-40ee-93f7-23241eb89469" (UID: "0151f1ec-56b3-40ee-93f7-23241eb89469"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:12:55.380865 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.380837    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0151f1ec-56b3-40ee-93f7-23241eb89469-kube-api-access-8sjxf" (OuterVolumeSpecName: "kube-api-access-8sjxf") pod "0151f1ec-56b3-40ee-93f7-23241eb89469" (UID: "0151f1ec-56b3-40ee-93f7-23241eb89469"). InnerVolumeSpecName "kube-api-access-8sjxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:12:55.480468 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.480436    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0151f1ec-56b3-40ee-93f7-23241eb89469-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:12:55.480468 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.480462    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sjxf\" (UniqueName: \"kubernetes.io/projected/0151f1ec-56b3-40ee-93f7-23241eb89469-kube-api-access-8sjxf\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:12:55.794165 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.794069    2549 generic.go:356] "Generic (PLEG): container finished" podID="0151f1ec-56b3-40ee-93f7-23241eb89469" containerID="306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b" exitCode=0
Jul 28 13:12:55.794165 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.794121    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"
Jul 28 13:12:55.794165 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.794149    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs" event={"ID":"0151f1ec-56b3-40ee-93f7-23241eb89469","Type":"ContainerDied","Data":"306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b"}
Jul 28 13:12:55.794445 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.794215    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs" event={"ID":"0151f1ec-56b3-40ee-93f7-23241eb89469","Type":"ContainerDied","Data":"1d6b0e4157795791ca4e6f4e841de3a1ab35690f3e1b6910b692cf6bb0bb3826"}
Jul 28 13:12:55.794445 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.794250    2549 scope.go:117] "RemoveContainer" containerID="306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b"
Jul 28 13:12:55.795667 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.795637    2549 generic.go:356] "Generic (PLEG): container finished" podID="590f1ae5-4537-433b-8849-1a4c13bf55cf" containerID="baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4" exitCode=0
Jul 28 13:12:55.795782 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.795677    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"
Jul 28 13:12:55.795782 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.795704    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl" event={"ID":"590f1ae5-4537-433b-8849-1a4c13bf55cf","Type":"ContainerDied","Data":"baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4"}
Jul 28 13:12:55.795782 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.795740    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl" event={"ID":"590f1ae5-4537-433b-8849-1a4c13bf55cf","Type":"ContainerDied","Data":"d4d8c54165175cead21a0045e5aa9df81f08506d860df8da41121ccea710e379"}
Jul 28 13:12:55.805461 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.805441    2549 scope.go:117] "RemoveContainer" containerID="306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b"
Jul 28 13:12:55.805851 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:12:55.805825    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b\": container with ID starting with 306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b not found: ID does not exist" containerID="306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b"
Jul 28 13:12:55.805945 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.805861    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b"} err="failed to get container status \"306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b\": rpc error: code = NotFound desc = could not find container \"306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b\": container with ID starting with 306a031a9a7cd7b8d7d893b158dd6d818bf4eb4d0ef429b32085d514d6ad413b not found: ID does not exist"
Jul 28 13:12:55.805945 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.805883    2549 scope.go:117] "RemoveContainer" containerID="baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4"
Jul 28 13:12:55.814718 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.814701    2549 scope.go:117] "RemoveContainer" containerID="baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4"
Jul 28 13:12:55.814973 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:12:55.814951    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4\": container with ID starting with baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4 not found: ID does not exist" containerID="baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4"
Jul 28 13:12:55.815049 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.814980    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4"} err="failed to get container status \"baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4\": rpc error: code = NotFound desc = could not find container \"baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4\": container with ID starting with baa8b1ecb7b40f1405c5e1eb206c0871a49037c050bbef2bd35a137ed07445e4 not found: ID does not exist"
Jul 28 13:12:55.821122 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.821098    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"]
Jul 28 13:12:55.828868 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.828849    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75cd7-57c69fb6f5-v86bs"]
Jul 28 13:12:55.841287 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.841265    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"]
Jul 28 13:12:55.843175 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:55.843156    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e75cd7-6b7c46fbb-j2ddl"]
Jul 28 13:12:56.692030 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:56.691995    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0151f1ec-56b3-40ee-93f7-23241eb89469" path="/var/lib/kubelet/pods/0151f1ec-56b3-40ee-93f7-23241eb89469/volumes"
Jul 28 13:12:56.692406 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:12:56.692346    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="590f1ae5-4537-433b-8849-1a4c13bf55cf" path="/var/lib/kubelet/pods/590f1ae5-4537-433b-8849-1a4c13bf55cf/volumes"
Jul 28 13:13:09.985354 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985322    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"]
Jul 28 13:13:09.985773 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985728    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="590f1ae5-4537-433b-8849-1a4c13bf55cf" containerName="payload-processing"
Jul 28 13:13:09.985773 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985739    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="590f1ae5-4537-433b-8849-1a4c13bf55cf" containerName="payload-processing"
Jul 28 13:13:09.985773 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985764    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4" containerName="maas-api"
Jul 28 13:13:09.985773 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985769    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4" containerName="maas-api"
Jul 28 13:13:09.985911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985777    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0151f1ec-56b3-40ee-93f7-23241eb89469" containerName="maas-api"
Jul 28 13:13:09.985911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985783    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0151f1ec-56b3-40ee-93f7-23241eb89469" containerName="maas-api"
Jul 28 13:13:09.985911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985839    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3cb9e4c-d936-4ab8-a2eb-714a54fbe9f4" containerName="maas-api"
Jul 28 13:13:09.985911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985849    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0151f1ec-56b3-40ee-93f7-23241eb89469" containerName="maas-api"
Jul 28 13:13:09.985911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.985857    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="590f1ae5-4537-433b-8849-1a4c13bf55cf" containerName="payload-processing"
Jul 28 13:13:09.991936 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.991916    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:09.995074 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.994881    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-12b3e2-serving-cert\""
Jul 28 13:13:09.997671 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:09.997646    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"]
Jul 28 13:13:10.024077 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.024031    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce20c437-ea89-45ab-857a-df93f2e7f394-openshift-service-ca\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.024230 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.024091    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce20c437-ea89-45ab-857a-df93f2e7f394-maas-api-tls\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.024230 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.024130    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgrd6\" (UniqueName: \"kubernetes.io/projected/ce20c437-ea89-45ab-857a-df93f2e7f394-kube-api-access-zgrd6\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.124849 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.124804    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce20c437-ea89-45ab-857a-df93f2e7f394-openshift-service-ca\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.125053 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.124863    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce20c437-ea89-45ab-857a-df93f2e7f394-maas-api-tls\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.125053 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.124898    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgrd6\" (UniqueName: \"kubernetes.io/projected/ce20c437-ea89-45ab-857a-df93f2e7f394-kube-api-access-zgrd6\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.125053 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:13:10.125023    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-serving-cert: secret "maas-api-e2e-api-a-12b3e2-serving-cert" not found
Jul 28 13:13:10.125220 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:13:10.125108    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce20c437-ea89-45ab-857a-df93f2e7f394-maas-api-tls podName:ce20c437-ea89-45ab-857a-df93f2e7f394 nodeName:}" failed. No retries permitted until 2026-07-28 13:13:10.625092004 +0000 UTC m=+2582.489341841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce20c437-ea89-45ab-857a-df93f2e7f394-maas-api-tls") pod "maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w" (UID: "ce20c437-ea89-45ab-857a-df93f2e7f394") : secret "maas-api-e2e-api-a-12b3e2-serving-cert" not found
Jul 28 13:13:10.125476 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.125455    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce20c437-ea89-45ab-857a-df93f2e7f394-openshift-service-ca\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.137345 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.137321    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgrd6\" (UniqueName: \"kubernetes.io/projected/ce20c437-ea89-45ab-857a-df93f2e7f394-kube-api-access-zgrd6\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.629540 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.629505    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce20c437-ea89-45ab-857a-df93f2e7f394-maas-api-tls\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.632189 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.632166    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce20c437-ea89-45ab-857a-df93f2e7f394-maas-api-tls\") pod \"maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:10.904451 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:10.904368    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:11.043374 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:11.043351    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"]
Jul 28 13:13:11.045577 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:13:11.045547    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce20c437_ea89_45ab_857a_df93f2e7f394.slice/crio-aad28caa85a9f24f5307ba31a73d7e392a8a723475d6f0662c3a25304021fc52 WatchSource:0}: Error finding container aad28caa85a9f24f5307ba31a73d7e392a8a723475d6f0662c3a25304021fc52: Status 404 returned error can't find the container with id aad28caa85a9f24f5307ba31a73d7e392a8a723475d6f0662c3a25304021fc52
Jul 28 13:13:11.046952 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:11.046932    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:13:11.884450 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:11.884409    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w" event={"ID":"ce20c437-ea89-45ab-857a-df93f2e7f394","Type":"ContainerStarted","Data":"d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6"}
Jul 28 13:13:11.884691 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:11.884469    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w" event={"ID":"ce20c437-ea89-45ab-857a-df93f2e7f394","Type":"ContainerStarted","Data":"aad28caa85a9f24f5307ba31a73d7e392a8a723475d6f0662c3a25304021fc52"}
Jul 28 13:13:11.884691 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:11.884578    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:11.906755 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:11.906706    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w" podStartSLOduration=2.533485319 podStartE2EDuration="2.906688198s" totalImagesPullingTime="373.202879ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:13:11.904807126 +0000 UTC m=+2583.769056985" watchObservedRunningTime="2026-07-28 13:13:11.906688198 +0000 UTC m=+2583.770938057"
Jul 28 13:13:17.895921 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:17.895892    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:18.029653 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.029603    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"]
Jul 28 13:13:18.034098 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.034074    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.038451 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.038428    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-bec5cf-serving-cert\""
Jul 28 13:13:18.047445 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.047417    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"]
Jul 28 13:13:18.103083 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.103044    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cccfbc-5449-40f8-a9a8-3293339e86df-openshift-service-ca\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.103255 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.103093    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.103255 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.103230    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57n6\" (UniqueName: \"kubernetes.io/projected/e5cccfbc-5449-40f8-a9a8-3293339e86df-kube-api-access-m57n6\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.204488 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.204380    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cccfbc-5449-40f8-a9a8-3293339e86df-openshift-service-ca\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.204488 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.204432    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.204771 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.204522    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57n6\" (UniqueName: \"kubernetes.io/projected/e5cccfbc-5449-40f8-a9a8-3293339e86df-kube-api-access-m57n6\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.204771 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:13:18.204619    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-serving-cert: secret "maas-api-e2e-api-b-bec5cf-serving-cert" not found
Jul 28 13:13:18.204771 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:13:18.204757    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls podName:e5cccfbc-5449-40f8-a9a8-3293339e86df nodeName:}" failed. No retries permitted until 2026-07-28 13:13:18.704732378 +0000 UTC m=+2590.568982234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls") pod "maas-api-e2e-api-b-bec5cf-7db47b659-d2rts" (UID: "e5cccfbc-5449-40f8-a9a8-3293339e86df") : secret "maas-api-e2e-api-b-bec5cf-serving-cert" not found
Jul 28 13:13:18.205180 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.205163    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cccfbc-5449-40f8-a9a8-3293339e86df-openshift-service-ca\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.216060 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.216032    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57n6\" (UniqueName: \"kubernetes.io/projected/e5cccfbc-5449-40f8-a9a8-3293339e86df-kube-api-access-m57n6\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.710446 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.710410    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:18.710609 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:13:18.710520    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-serving-cert: secret "maas-api-e2e-api-b-bec5cf-serving-cert" not found
Jul 28 13:13:18.710609 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:13:18.710591    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls podName:e5cccfbc-5449-40f8-a9a8-3293339e86df nodeName:}" failed. No retries permitted until 2026-07-28 13:13:19.710577386 +0000 UTC m=+2591.574827224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls") pod "maas-api-e2e-api-b-bec5cf-7db47b659-d2rts" (UID: "e5cccfbc-5449-40f8-a9a8-3293339e86df") : secret "maas-api-e2e-api-b-bec5cf-serving-cert" not found
Jul 28 13:13:18.755008 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.754974    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"]
Jul 28 13:13:18.758644 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.758598    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:18.761745 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.761721    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-bec5cf\""
Jul 28 13:13:18.761892 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.761793    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-bec5cf-dockercfg-l8mhf\""
Jul 28 13:13:18.770087 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.770050    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"]
Jul 28 13:13:18.811321 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.811284    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fdca1c1-12b7-4700-b096-fdd189a89df7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr\" (UID: \"2fdca1c1-12b7-4700-b096-fdd189a89df7\") " pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:18.811485 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.811395    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrmws\" (UniqueName: \"kubernetes.io/projected/2fdca1c1-12b7-4700-b096-fdd189a89df7-kube-api-access-zrmws\") pod \"payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr\" (UID: \"2fdca1c1-12b7-4700-b096-fdd189a89df7\") " pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:18.912064 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.912021    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrmws\" (UniqueName: \"kubernetes.io/projected/2fdca1c1-12b7-4700-b096-fdd189a89df7-kube-api-access-zrmws\") pod \"payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr\" (UID: \"2fdca1c1-12b7-4700-b096-fdd189a89df7\") " pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:18.912492 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.912142    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fdca1c1-12b7-4700-b096-fdd189a89df7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr\" (UID: \"2fdca1c1-12b7-4700-b096-fdd189a89df7\") " pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:18.913002 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.912979    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fdca1c1-12b7-4700-b096-fdd189a89df7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr\" (UID: \"2fdca1c1-12b7-4700-b096-fdd189a89df7\") " pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:18.922402 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:18.922371    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrmws\" (UniqueName: \"kubernetes.io/projected/2fdca1c1-12b7-4700-b096-fdd189a89df7-kube-api-access-zrmws\") pod \"payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr\" (UID: \"2fdca1c1-12b7-4700-b096-fdd189a89df7\") " pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:19.072213 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.072183    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:19.221353 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.221321    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"]
Jul 28 13:13:19.224452 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:13:19.224415    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fdca1c1_12b7_4700_b096_fdd189a89df7.slice/crio-4299ee738aceea441bd05dc9f59531847154bf4e961acc15b23a0c7fd8aad151 WatchSource:0}: Error finding container 4299ee738aceea441bd05dc9f59531847154bf4e961acc15b23a0c7fd8aad151: Status 404 returned error can't find the container with id 4299ee738aceea441bd05dc9f59531847154bf4e961acc15b23a0c7fd8aad151
Jul 28 13:13:19.718799 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.718758    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:19.721420 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.721385    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls\") pod \"maas-api-e2e-api-b-bec5cf-7db47b659-d2rts\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:19.847444 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.847406    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:19.929503 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.929453    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr" event={"ID":"2fdca1c1-12b7-4700-b096-fdd189a89df7","Type":"ContainerStarted","Data":"31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8"}
Jul 28 13:13:19.929916 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.929519    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr" event={"ID":"2fdca1c1-12b7-4700-b096-fdd189a89df7","Type":"ContainerStarted","Data":"4299ee738aceea441bd05dc9f59531847154bf4e961acc15b23a0c7fd8aad151"}
Jul 28 13:13:19.929916 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.929618    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:19.947454 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.947402    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr" podStartSLOduration=1.947387715 podStartE2EDuration="1.947387715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:13:19.945705061 +0000 UTC m=+2591.809954920" watchObservedRunningTime="2026-07-28 13:13:19.947387715 +0000 UTC m=+2591.811637573"
Jul 28 13:13:19.989978 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:19.989951    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"]
Jul 28 13:13:19.992194 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:13:19.992167    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5cccfbc_5449_40f8_a9a8_3293339e86df.slice/crio-70614ea36c3b89bbd766a7a0a3e5f86a11dfebbd3a8c5f0cd9f32bb892e0640b WatchSource:0}: Error finding container 70614ea36c3b89bbd766a7a0a3e5f86a11dfebbd3a8c5f0cd9f32bb892e0640b: Status 404 returned error can't find the container with id 70614ea36c3b89bbd766a7a0a3e5f86a11dfebbd3a8c5f0cd9f32bb892e0640b
Jul 28 13:13:20.937261 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:20.937209    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts" event={"ID":"e5cccfbc-5449-40f8-a9a8-3293339e86df","Type":"ContainerStarted","Data":"2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e"}
Jul 28 13:13:20.937771 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:20.937279    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts" event={"ID":"e5cccfbc-5449-40f8-a9a8-3293339e86df","Type":"ContainerStarted","Data":"70614ea36c3b89bbd766a7a0a3e5f86a11dfebbd3a8c5f0cd9f32bb892e0640b"}
Jul 28 13:13:20.937771 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:20.937315    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:20.970524 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:20.970459    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts" podStartSLOduration=2.144329065 podStartE2EDuration="2.970437176s" totalImagesPullingTime="826.108111ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:13:20.968745377 +0000 UTC m=+2592.832995239" watchObservedRunningTime="2026-07-28 13:13:20.970437176 +0000 UTC m=+2592.834687036"
Jul 28 13:13:26.950682 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:26.950651    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:13:30.940441 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:30.940349    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:13:47.057350 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.057313    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"]
Jul 28 13:13:47.057977 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.057540    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w" podUID="ce20c437-ea89-45ab-857a-df93f2e7f394" containerName="maas-api" containerID="cri-o://d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6" gracePeriod=30
Jul 28 13:13:47.320233 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.320163    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:47.379589 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.379550    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce20c437-ea89-45ab-857a-df93f2e7f394-openshift-service-ca\") pod \"ce20c437-ea89-45ab-857a-df93f2e7f394\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") "
Jul 28 13:13:47.379807 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.379600    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgrd6\" (UniqueName: \"kubernetes.io/projected/ce20c437-ea89-45ab-857a-df93f2e7f394-kube-api-access-zgrd6\") pod \"ce20c437-ea89-45ab-857a-df93f2e7f394\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") "
Jul 28 13:13:47.379807 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.379665    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce20c437-ea89-45ab-857a-df93f2e7f394-maas-api-tls\") pod \"ce20c437-ea89-45ab-857a-df93f2e7f394\" (UID: \"ce20c437-ea89-45ab-857a-df93f2e7f394\") "
Jul 28 13:13:47.379969 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.379939    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce20c437-ea89-45ab-857a-df93f2e7f394-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce20c437-ea89-45ab-857a-df93f2e7f394" (UID: "ce20c437-ea89-45ab-857a-df93f2e7f394"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:13:47.381958 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.381928    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce20c437-ea89-45ab-857a-df93f2e7f394-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce20c437-ea89-45ab-857a-df93f2e7f394" (UID: "ce20c437-ea89-45ab-857a-df93f2e7f394"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:13:47.382097 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.381994    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce20c437-ea89-45ab-857a-df93f2e7f394-kube-api-access-zgrd6" (OuterVolumeSpecName: "kube-api-access-zgrd6") pod "ce20c437-ea89-45ab-857a-df93f2e7f394" (UID: "ce20c437-ea89-45ab-857a-df93f2e7f394"). InnerVolumeSpecName "kube-api-access-zgrd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:13:47.481416 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.481379    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce20c437-ea89-45ab-857a-df93f2e7f394-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:13:47.481416 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.481407    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgrd6\" (UniqueName: \"kubernetes.io/projected/ce20c437-ea89-45ab-857a-df93f2e7f394-kube-api-access-zgrd6\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:13:47.481416 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:47.481417    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce20c437-ea89-45ab-857a-df93f2e7f394-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:13:48.086382 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.086350    2549 generic.go:356] "Generic (PLEG): container finished" podID="ce20c437-ea89-45ab-857a-df93f2e7f394" containerID="d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6" exitCode=0
Jul 28 13:13:48.086876 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.086404    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"
Jul 28 13:13:48.086876 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.086420    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w" event={"ID":"ce20c437-ea89-45ab-857a-df93f2e7f394","Type":"ContainerDied","Data":"d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6"}
Jul 28 13:13:48.086876 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.086476    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w" event={"ID":"ce20c437-ea89-45ab-857a-df93f2e7f394","Type":"ContainerDied","Data":"aad28caa85a9f24f5307ba31a73d7e392a8a723475d6f0662c3a25304021fc52"}
Jul 28 13:13:48.086876 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.086508    2549 scope.go:117] "RemoveContainer" containerID="d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6"
Jul 28 13:13:48.096312 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.096294    2549 scope.go:117] "RemoveContainer" containerID="d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6"
Jul 28 13:13:48.096561 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:13:48.096542    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6\": container with ID starting with d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6 not found: ID does not exist" containerID="d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6"
Jul 28 13:13:48.096656 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.096570    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6"} err="failed to get container status \"d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6\": rpc error: code = NotFound desc = could not find container \"d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6\": container with ID starting with d5a697592c1e04b7ea90389c69484ce3db8e4ea594dc8114152796974b0186f6 not found: ID does not exist"
Jul 28 13:13:48.109517 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.109497    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"]
Jul 28 13:13:48.112784 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.112765    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-12b3e2-56fd9fd49f-6mp2w"]
Jul 28 13:13:48.692290 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:13:48.692248    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce20c437-ea89-45ab-857a-df93f2e7f394" path="/var/lib/kubelet/pods/ce20c437-ea89-45ab-857a-df93f2e7f394/volumes"
Jul 28 13:14:13.715208 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:13.715168    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"]
Jul 28 13:14:13.715704 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:13.715442    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts" podUID="e5cccfbc-5449-40f8-a9a8-3293339e86df" containerName="maas-api" containerID="cri-o://2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e" gracePeriod=30
Jul 28 13:14:13.768046 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:13.767999    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"]
Jul 28 13:14:13.768355 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:13.768310    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr" podUID="2fdca1c1-12b7-4700-b096-fdd189a89df7" containerName="payload-processing" containerID="cri-o://31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8" gracePeriod=30
Jul 28 13:14:14.045951 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.045925    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:14:14.065266 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.065222    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:14:14.137598 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.137569    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cccfbc-5449-40f8-a9a8-3293339e86df-openshift-service-ca\") pod \"e5cccfbc-5449-40f8-a9a8-3293339e86df\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") "
Jul 28 13:14:14.137790 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.137720    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m57n6\" (UniqueName: \"kubernetes.io/projected/e5cccfbc-5449-40f8-a9a8-3293339e86df-kube-api-access-m57n6\") pod \"e5cccfbc-5449-40f8-a9a8-3293339e86df\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") "
Jul 28 13:14:14.137790 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.137762    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls\") pod \"e5cccfbc-5449-40f8-a9a8-3293339e86df\" (UID: \"e5cccfbc-5449-40f8-a9a8-3293339e86df\") "
Jul 28 13:14:14.137998 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.137975    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5cccfbc-5449-40f8-a9a8-3293339e86df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5cccfbc-5449-40f8-a9a8-3293339e86df" (UID: "e5cccfbc-5449-40f8-a9a8-3293339e86df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:14:14.138081 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.138063    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cccfbc-5449-40f8-a9a8-3293339e86df-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:14:14.139884 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.139858    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5cccfbc-5449-40f8-a9a8-3293339e86df" (UID: "e5cccfbc-5449-40f8-a9a8-3293339e86df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:14:14.140001 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.139978    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5cccfbc-5449-40f8-a9a8-3293339e86df-kube-api-access-m57n6" (OuterVolumeSpecName: "kube-api-access-m57n6") pod "e5cccfbc-5449-40f8-a9a8-3293339e86df" (UID: "e5cccfbc-5449-40f8-a9a8-3293339e86df"). InnerVolumeSpecName "kube-api-access-m57n6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:14:14.225697 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.225582    2549 generic.go:356] "Generic (PLEG): container finished" podID="2fdca1c1-12b7-4700-b096-fdd189a89df7" containerID="31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8" exitCode=0
Jul 28 13:14:14.225697 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.225653    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"
Jul 28 13:14:14.225895 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.225687    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr" event={"ID":"2fdca1c1-12b7-4700-b096-fdd189a89df7","Type":"ContainerDied","Data":"31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8"}
Jul 28 13:14:14.225895 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.225784    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr" event={"ID":"2fdca1c1-12b7-4700-b096-fdd189a89df7","Type":"ContainerDied","Data":"4299ee738aceea441bd05dc9f59531847154bf4e961acc15b23a0c7fd8aad151"}
Jul 28 13:14:14.225895 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.225825    2549 scope.go:117] "RemoveContainer" containerID="31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8"
Jul 28 13:14:14.227048 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.226960    2549 generic.go:356] "Generic (PLEG): container finished" podID="e5cccfbc-5449-40f8-a9a8-3293339e86df" containerID="2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e" exitCode=0
Jul 28 13:14:14.227048 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.227003    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"
Jul 28 13:14:14.227158 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.227083    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts" event={"ID":"e5cccfbc-5449-40f8-a9a8-3293339e86df","Type":"ContainerDied","Data":"2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e"}
Jul 28 13:14:14.227158 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.227120    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts" event={"ID":"e5cccfbc-5449-40f8-a9a8-3293339e86df","Type":"ContainerDied","Data":"70614ea36c3b89bbd766a7a0a3e5f86a11dfebbd3a8c5f0cd9f32bb892e0640b"}
Jul 28 13:14:14.234869 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.234849    2549 scope.go:117] "RemoveContainer" containerID="31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8"
Jul 28 13:14:14.235130 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:14:14.235107    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8\": container with ID starting with 31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8 not found: ID does not exist" containerID="31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8"
Jul 28 13:14:14.235232 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.235139    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8"} err="failed to get container status \"31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8\": rpc error: code = NotFound desc = could not find container \"31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8\": container with ID starting with 31457bcd7a3deb66829eef20f17c5146222578e69e1df3d59b0d9f2830e709e8 not found: ID does not exist"
Jul 28 13:14:14.235232 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.235159    2549 scope.go:117] "RemoveContainer" containerID="2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e"
Jul 28 13:14:14.238347 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.238323    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrmws\" (UniqueName: \"kubernetes.io/projected/2fdca1c1-12b7-4700-b096-fdd189a89df7-kube-api-access-zrmws\") pod \"2fdca1c1-12b7-4700-b096-fdd189a89df7\" (UID: \"2fdca1c1-12b7-4700-b096-fdd189a89df7\") "
Jul 28 13:14:14.238441 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.238396    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fdca1c1-12b7-4700-b096-fdd189a89df7-plugins-config-volume\") pod \"2fdca1c1-12b7-4700-b096-fdd189a89df7\" (UID: \"2fdca1c1-12b7-4700-b096-fdd189a89df7\") "
Jul 28 13:14:14.238785 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.238756    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fdca1c1-12b7-4700-b096-fdd189a89df7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2fdca1c1-12b7-4700-b096-fdd189a89df7" (UID: "2fdca1c1-12b7-4700-b096-fdd189a89df7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:14:14.238886 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.238794    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m57n6\" (UniqueName: \"kubernetes.io/projected/e5cccfbc-5449-40f8-a9a8-3293339e86df-kube-api-access-m57n6\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:14:14.238886 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.238814    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5cccfbc-5449-40f8-a9a8-3293339e86df-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:14:14.240437 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.240418    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fdca1c1-12b7-4700-b096-fdd189a89df7-kube-api-access-zrmws" (OuterVolumeSpecName: "kube-api-access-zrmws") pod "2fdca1c1-12b7-4700-b096-fdd189a89df7" (UID: "2fdca1c1-12b7-4700-b096-fdd189a89df7"). InnerVolumeSpecName "kube-api-access-zrmws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:14:14.243894 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.243870    2549 scope.go:117] "RemoveContainer" containerID="2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e"
Jul 28 13:14:14.244171 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:14:14.244152    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e\": container with ID starting with 2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e not found: ID does not exist" containerID="2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e"
Jul 28 13:14:14.244243 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.244181    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e"} err="failed to get container status \"2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e\": rpc error: code = NotFound desc = could not find container \"2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e\": container with ID starting with 2917b4fce4290847ca971ca4374e5b9fbdae4062200dacc165958fb09ba7511e not found: ID does not exist"
Jul 28 13:14:14.248979 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.248959    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"]
Jul 28 13:14:14.252989 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.252970    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bec5cf-7db47b659-d2rts"]
Jul 28 13:14:14.340203 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.340178    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrmws\" (UniqueName: \"kubernetes.io/projected/2fdca1c1-12b7-4700-b096-fdd189a89df7-kube-api-access-zrmws\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:14:14.340203 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.340202    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fdca1c1-12b7-4700-b096-fdd189a89df7-plugins-config-volume\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:14:14.549199 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.549167    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"]
Jul 28 13:14:14.554437 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.554412    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bec5cf-56c8745d6b-ldlzr"]
Jul 28 13:14:14.692758 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.692726    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fdca1c1-12b7-4700-b096-fdd189a89df7" path="/var/lib/kubelet/pods/2fdca1c1-12b7-4700-b096-fdd189a89df7/volumes"
Jul 28 13:14:14.693163 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:14.693149    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5cccfbc-5449-40f8-a9a8-3293339e86df" path="/var/lib/kubelet/pods/e5cccfbc-5449-40f8-a9a8-3293339e86df/volumes"
Jul 28 13:14:28.578599 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.578556    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"]
Jul 28 13:14:28.579185 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.579165    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fdca1c1-12b7-4700-b096-fdd189a89df7" containerName="payload-processing"
Jul 28 13:14:28.579242 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.579189    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fdca1c1-12b7-4700-b096-fdd189a89df7" containerName="payload-processing"
Jul 28 13:14:28.579242 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.579222    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce20c437-ea89-45ab-857a-df93f2e7f394" containerName="maas-api"
Jul 28 13:14:28.579242 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.579231    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce20c437-ea89-45ab-857a-df93f2e7f394" containerName="maas-api"
Jul 28 13:14:28.579335 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.579249    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5cccfbc-5449-40f8-a9a8-3293339e86df" containerName="maas-api"
Jul 28 13:14:28.579335 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.579259    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5cccfbc-5449-40f8-a9a8-3293339e86df" containerName="maas-api"
Jul 28 13:14:28.579401 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.579354    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5cccfbc-5449-40f8-a9a8-3293339e86df" containerName="maas-api"
Jul 28 13:14:28.579401 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.579368    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce20c437-ea89-45ab-857a-df93f2e7f394" containerName="maas-api"
Jul 28 13:14:28.579401 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.579380    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fdca1c1-12b7-4700-b096-fdd189a89df7" containerName="payload-processing"
Jul 28 13:14:28.584497 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.584474    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.589464 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.589444    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-67d68f-serving-cert\""
Jul 28 13:14:28.598811 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.598791    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"]
Jul 28 13:14:28.674543 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.674511    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8gnn\" (UniqueName: \"kubernetes.io/projected/079edf1c-9dc1-44e0-accb-c98b172d4bf6-kube-api-access-g8gnn\") pod \"maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.674720 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.674653    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079edf1c-9dc1-44e0-accb-c98b172d4bf6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.674720 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.674708    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079edf1c-9dc1-44e0-accb-c98b172d4bf6-maas-api-tls\") pod \"maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.776112 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.776076    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079edf1c-9dc1-44e0-accb-c98b172d4bf6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.776309 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.776133    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079edf1c-9dc1-44e0-accb-c98b172d4bf6-maas-api-tls\") pod \"maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.776309 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.776168    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8gnn\" (UniqueName: \"kubernetes.io/projected/079edf1c-9dc1-44e0-accb-c98b172d4bf6-kube-api-access-g8gnn\") pod \"maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.776957 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.776905    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079edf1c-9dc1-44e0-accb-c98b172d4bf6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.779016 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.778991    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079edf1c-9dc1-44e0-accb-c98b172d4bf6-maas-api-tls\") pod \"maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.785279 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.785255    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8gnn\" (UniqueName: \"kubernetes.io/projected/079edf1c-9dc1-44e0-accb-c98b172d4bf6-kube-api-access-g8gnn\") pod \"maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:28.895048 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:28.894940    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:29.036747 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:29.036720    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"]
Jul 28 13:14:29.039370 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:14:29.039339    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079edf1c_9dc1_44e0_accb_c98b172d4bf6.slice/crio-cd16fece80dd5e2a922ab741df031a1240e6880a293fc2fc252b4a787062fad4 WatchSource:0}: Error finding container cd16fece80dd5e2a922ab741df031a1240e6880a293fc2fc252b4a787062fad4: Status 404 returned error can't find the container with id cd16fece80dd5e2a922ab741df031a1240e6880a293fc2fc252b4a787062fad4
Jul 28 13:14:29.306932 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:29.306891    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg" event={"ID":"079edf1c-9dc1-44e0-accb-c98b172d4bf6","Type":"ContainerStarted","Data":"cd16fece80dd5e2a922ab741df031a1240e6880a293fc2fc252b4a787062fad4"}
Jul 28 13:14:30.313710 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:30.313667    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg" event={"ID":"079edf1c-9dc1-44e0-accb-c98b172d4bf6","Type":"ContainerStarted","Data":"2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f"}
Jul 28 13:14:30.314083 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:30.313736    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:30.331112 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:30.331066    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg" podStartSLOduration=1.919214843 podStartE2EDuration="2.331051467s" totalImagesPullingTime="411.836624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:14:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:14:30.329941042 +0000 UTC m=+2662.194190902" watchObservedRunningTime="2026-07-28 13:14:30.331051467 +0000 UTC m=+2662.195301326"
Jul 28 13:14:36.324479 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.324447    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:14:36.426045 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.426001    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"]
Jul 28 13:14:36.429869 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.429846    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:36.432892 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.432865    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-827464-serving-cert\""
Jul 28 13:14:36.442057 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.442029    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"]
Jul 28 13:14:36.548702 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.548659    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgf9n\" (UniqueName: \"kubernetes.io/projected/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-kube-api-access-pgf9n\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:36.549021 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.548816    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:36.549021 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.548882    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-maas-api-tls\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:36.650007 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.649902    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgf9n\" (UniqueName: \"kubernetes.io/projected/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-kube-api-access-pgf9n\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:36.650007 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.649995    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:36.650266 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.650031    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-maas-api-tls\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:36.650266 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:14:36.650142    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-serving-cert: secret "maas-api-e2e-shared-b-827464-serving-cert" not found
Jul 28 13:14:36.650266 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:14:36.650229    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-maas-api-tls podName:2e31ec89-4055-4b25-a0c6-3ed80acaa14f nodeName:}" failed. No retries permitted until 2026-07-28 13:14:37.150205806 +0000 UTC m=+2669.014455643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-maas-api-tls") pod "maas-api-e2e-shared-b-827464-646b444f77-lsjxr" (UID: "2e31ec89-4055-4b25-a0c6-3ed80acaa14f") : secret "maas-api-e2e-shared-b-827464-serving-cert" not found
Jul 28 13:14:36.650681 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.650661    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:36.659328 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:36.659305    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgf9n\" (UniqueName: \"kubernetes.io/projected/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-kube-api-access-pgf9n\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:37.155733 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.155698    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-maas-api-tls\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:37.158324 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.158306    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-maas-api-tls\") pod \"maas-api-e2e-shared-b-827464-646b444f77-lsjxr\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:37.183232 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.183200    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"]
Jul 28 13:14:37.187202 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.187177    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:14:37.190178 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.190147    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-827464-dockercfg-rcf8r\""
Jul 28 13:14:37.194437 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.194412    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-827464\""
Jul 28 13:14:37.202316 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.202284    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"]
Jul 28 13:14:37.257260 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.257224    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhxqb\" (UniqueName: \"kubernetes.io/projected/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-kube-api-access-mhxqb\") pod \"payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2\" (UID: \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:14:37.257460 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.257295    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2\" (UID: \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:14:37.342360 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.342318    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:37.358100 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.358071    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2\" (UID: \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:14:37.358258 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.358166    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhxqb\" (UniqueName: \"kubernetes.io/projected/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-kube-api-access-mhxqb\") pod \"payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2\" (UID: \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:14:37.358839 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.358813    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2\" (UID: \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:14:37.367758 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.367730    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhxqb\" (UniqueName: \"kubernetes.io/projected/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-kube-api-access-mhxqb\") pod \"payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2\" (UID: \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:14:37.489967 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.489937    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"]
Jul 28 13:14:37.492899 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:14:37.492872    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e31ec89_4055_4b25_a0c6_3ed80acaa14f.slice/crio-67c18c3d3d662b7b4140c35cc93a9d41a378c5591e73ab3948324d1693f2cd3c WatchSource:0}: Error finding container 67c18c3d3d662b7b4140c35cc93a9d41a378c5591e73ab3948324d1693f2cd3c: Status 404 returned error can't find the container with id 67c18c3d3d662b7b4140c35cc93a9d41a378c5591e73ab3948324d1693f2cd3c
Jul 28 13:14:37.499287 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.499262    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:14:37.642494 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:37.642461    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"]
Jul 28 13:14:37.644107 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:14:37.644080    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a23b0a_3b17_48f2_a25e_0944e34cc6fe.slice/crio-d856074b9f07c8083068d9b0732e98b9107524045e9f734b5ba3321f7c4a7036 WatchSource:0}: Error finding container d856074b9f07c8083068d9b0732e98b9107524045e9f734b5ba3321f7c4a7036: Status 404 returned error can't find the container with id d856074b9f07c8083068d9b0732e98b9107524045e9f734b5ba3321f7c4a7036
Jul 28 13:14:38.354402 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:38.354363    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr" event={"ID":"2e31ec89-4055-4b25-a0c6-3ed80acaa14f","Type":"ContainerStarted","Data":"3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93"}
Jul 28 13:14:38.354885 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:38.354432    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:38.354885 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:38.354454    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr" event={"ID":"2e31ec89-4055-4b25-a0c6-3ed80acaa14f","Type":"ContainerStarted","Data":"67c18c3d3d662b7b4140c35cc93a9d41a378c5591e73ab3948324d1693f2cd3c"}
Jul 28 13:14:38.355990 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:38.355964    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2" event={"ID":"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe","Type":"ContainerStarted","Data":"9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1"}
Jul 28 13:14:38.356100 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:38.356009    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2" event={"ID":"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe","Type":"ContainerStarted","Data":"d856074b9f07c8083068d9b0732e98b9107524045e9f734b5ba3321f7c4a7036"}
Jul 28 13:14:38.356100 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:38.356087    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:14:38.373445 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:38.373384    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr" podStartSLOduration=2.015140917 podStartE2EDuration="2.373365934s" totalImagesPullingTime="358.225017ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:14:38.370903389 +0000 UTC m=+2670.235153248" watchObservedRunningTime="2026-07-28 13:14:38.373365934 +0000 UTC m=+2670.237615794"
Jul 28 13:14:38.387684 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:38.387640    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2" podStartSLOduration=1.387608832 podStartE2EDuration="1.387608832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:14:38.386949992 +0000 UTC m=+2670.251199853" watchObservedRunningTime="2026-07-28 13:14:38.387608832 +0000 UTC m=+2670.251858725"
Jul 28 13:14:44.366598 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:44.366557    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:14:49.363501 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:14:49.363472    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:15:08.783198 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:15:08.783102    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:15:08.800058 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:15:08.800033    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:20:08.818711 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:20:08.818685    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:20:08.832348 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:20:08.832328    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:23:54.599032 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.598984    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"]
Jul 28 13:23:54.602818 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.602792    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.606176 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.606149    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-8fb56b-serving-cert\""
Jul 28 13:23:54.630066 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.630040    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"]
Jul 28 13:23:54.737026 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.736992    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1390502d-0e9e-4dee-9840-1a154672c9ce-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.737026 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.737026    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1390502d-0e9e-4dee-9840-1a154672c9ce-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.737250 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.737077    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxhj7\" (UniqueName: \"kubernetes.io/projected/1390502d-0e9e-4dee-9840-1a154672c9ce-kube-api-access-fxhj7\") pod \"maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.838425 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.838392    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1390502d-0e9e-4dee-9840-1a154672c9ce-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.838425 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.838427    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1390502d-0e9e-4dee-9840-1a154672c9ce-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.838721 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.838695    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxhj7\" (UniqueName: \"kubernetes.io/projected/1390502d-0e9e-4dee-9840-1a154672c9ce-kube-api-access-fxhj7\") pod \"maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.839224 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.839200    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1390502d-0e9e-4dee-9840-1a154672c9ce-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.841110 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.841089    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1390502d-0e9e-4dee-9840-1a154672c9ce-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.847320 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.847298    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxhj7\" (UniqueName: \"kubernetes.io/projected/1390502d-0e9e-4dee-9840-1a154672c9ce-kube-api-access-fxhj7\") pod \"maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:54.913010 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:54.912938    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:55.055962 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.055933    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"]
Jul 28 13:23:55.058534 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:23:55.058510    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1390502d_0e9e_4dee_9840_1a154672c9ce.slice/crio-2c162a2a2f4bfc316fba5c0eb6377813003bdc4442765b02847a166e9ecd6cbd WatchSource:0}: Error finding container 2c162a2a2f4bfc316fba5c0eb6377813003bdc4442765b02847a166e9ecd6cbd: Status 404 returned error can't find the container with id 2c162a2a2f4bfc316fba5c0eb6377813003bdc4442765b02847a166e9ecd6cbd
Jul 28 13:23:55.059944 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.059926    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 13:23:55.246383 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.246294    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn" event={"ID":"1390502d-0e9e-4dee-9840-1a154672c9ce","Type":"ContainerStarted","Data":"2c162a2a2f4bfc316fba5c0eb6377813003bdc4442765b02847a166e9ecd6cbd"}
Jul 28 13:23:55.261414 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.261377    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"]
Jul 28 13:23:55.266390 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.266373    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:23:55.269532 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.269507    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-8fb56b\""
Jul 28 13:23:55.269695 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.269507    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-8fb56b-dockercfg-ccnx4\""
Jul 28 13:23:55.279156 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.279135    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"]
Jul 28 13:23:55.344287 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.344255    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jgtn\" (UniqueName: \"kubernetes.io/projected/906297b7-e672-4b85-9d92-2146b1490975-kube-api-access-8jgtn\") pod \"payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4\" (UID: \"906297b7-e672-4b85-9d92-2146b1490975\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:23:55.344481 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.344323    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906297b7-e672-4b85-9d92-2146b1490975-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4\" (UID: \"906297b7-e672-4b85-9d92-2146b1490975\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:23:55.445959 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.445919    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jgtn\" (UniqueName: \"kubernetes.io/projected/906297b7-e672-4b85-9d92-2146b1490975-kube-api-access-8jgtn\") pod \"payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4\" (UID: \"906297b7-e672-4b85-9d92-2146b1490975\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:23:55.445959 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.445967    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906297b7-e672-4b85-9d92-2146b1490975-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4\" (UID: \"906297b7-e672-4b85-9d92-2146b1490975\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:23:55.446771 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.446752    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906297b7-e672-4b85-9d92-2146b1490975-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4\" (UID: \"906297b7-e672-4b85-9d92-2146b1490975\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:23:55.454468 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.454446    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jgtn\" (UniqueName: \"kubernetes.io/projected/906297b7-e672-4b85-9d92-2146b1490975-kube-api-access-8jgtn\") pod \"payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4\" (UID: \"906297b7-e672-4b85-9d92-2146b1490975\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:23:55.579027 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.578987    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:23:55.722355 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:55.722210    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"]
Jul 28 13:23:55.724755 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:23:55.724728    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906297b7_e672_4b85_9d92_2146b1490975.slice/crio-e54516e44890868cc27e8f199f862621e488712e96bf8802d2469a005e7b2c52 WatchSource:0}: Error finding container e54516e44890868cc27e8f199f862621e488712e96bf8802d2469a005e7b2c52: Status 404 returned error can't find the container with id e54516e44890868cc27e8f199f862621e488712e96bf8802d2469a005e7b2c52
Jul 28 13:23:56.254027 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:56.253987    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4" event={"ID":"906297b7-e672-4b85-9d92-2146b1490975","Type":"ContainerStarted","Data":"a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d"}
Jul 28 13:23:56.254251 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:56.254062    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:23:56.254251 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:56.254087    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4" event={"ID":"906297b7-e672-4b85-9d92-2146b1490975","Type":"ContainerStarted","Data":"e54516e44890868cc27e8f199f862621e488712e96bf8802d2469a005e7b2c52"}
Jul 28 13:23:56.255368 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:56.255343    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn" event={"ID":"1390502d-0e9e-4dee-9840-1a154672c9ce","Type":"ContainerStarted","Data":"2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb"}
Jul 28 13:23:56.255496 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:56.255480    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:23:56.273964 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:56.273908    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4" podStartSLOduration=1.2738896579999999 podStartE2EDuration="1.273889658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:23:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:23:56.273458422 +0000 UTC m=+3228.137708285" watchObservedRunningTime="2026-07-28 13:23:56.273889658 +0000 UTC m=+3228.138139520"
Jul 28 13:23:56.298510 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:23:56.298461    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn" podStartSLOduration=1.585104157 podStartE2EDuration="2.298446343s" totalImagesPullingTime="713.342186ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:23:56.295997191 +0000 UTC m=+3228.160247051" watchObservedRunningTime="2026-07-28 13:23:56.298446343 +0000 UTC m=+3228.162696246"
Jul 28 13:24:02.267846 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:02.267763    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:24:07.263823 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:07.263792    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:24:10.120551 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.120519    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"]
Jul 28 13:24:10.125440 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.125409    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.128585 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.128559    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-06a4d2-openshift-default-dockercfg-rqg7k\""
Jul 28 13:24:10.137205 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.137180    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"]
Jul 28 13:24:10.289449 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.289403    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb1f775-25ca-439c-9e5c-660f7703e36a-istiod-ca-cert\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.289449 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.289444    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-podinfo\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.289672 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.289474    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-certs\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.289672 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.289510    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-data\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.289672 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.289531    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-credential-socket\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.289672 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.289657    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zd6v\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-kube-api-access-7zd6v\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.289806 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.289692    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-socket\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.289806 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.289725    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-token\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.289806 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.289748    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-envoy\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.390988 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.390893    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zd6v\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-kube-api-access-7zd6v\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.390988 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.390956    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-socket\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.390988 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.390985    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-token\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391279 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391016    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-envoy\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391279 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391105    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb1f775-25ca-439c-9e5c-660f7703e36a-istiod-ca-cert\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391279 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391133    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-podinfo\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391279 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391163    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-certs\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391279 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391215    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-data\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391279 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391251    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-credential-socket\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391557 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391490    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-socket\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391751 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391728    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-credential-socket\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391844 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391790    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-data\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.391844 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.391824    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-certs\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.392128 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.392107    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb1f775-25ca-439c-9e5c-660f7703e36a-istiod-ca-cert\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.393521 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.393499    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-envoy\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.393768 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.393750    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-podinfo\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.400403 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.400373    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-token\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.400589 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.400573    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zd6v\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-kube-api-access-7zd6v\") pod \"e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") " pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.438357 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.438331    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:10.585391 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.585360    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"]
Jul 28 13:24:10.586217 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:24:10.586190    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb1f775_25ca_439c_9e5c_660f7703e36a.slice/crio-d8c2a91dccfbc2e0a0f3359208520cc76a006fe7443d23de86ed88de6c4060ca WatchSource:0}: Error finding container d8c2a91dccfbc2e0a0f3359208520cc76a006fe7443d23de86ed88de6c4060ca: Status 404 returned error can't find the container with id d8c2a91dccfbc2e0a0f3359208520cc76a006fe7443d23de86ed88de6c4060ca
Jul 28 13:24:10.588490 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.588458    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:24:10.588585 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.588524    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:24:10.588585 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:10.588556    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:24:11.341995 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:11.341955    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" event={"ID":"bdb1f775-25ca-439c-9e5c-660f7703e36a","Type":"ContainerStarted","Data":"f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9"}
Jul 28 13:24:11.342431 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:11.342009    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" event={"ID":"bdb1f775-25ca-439c-9e5c-660f7703e36a","Type":"ContainerStarted","Data":"d8c2a91dccfbc2e0a0f3359208520cc76a006fe7443d23de86ed88de6c4060ca"}
Jul 28 13:24:11.365419 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:11.365361    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" podStartSLOduration=1.365347314 podStartE2EDuration="1.365347314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:24:11.364270088 +0000 UTC m=+3243.228519958" watchObservedRunningTime="2026-07-28 13:24:11.365347314 +0000 UTC m=+3243.229597172"
Jul 28 13:24:11.438828 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:11.438797    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:11.443339 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:11.443314    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:12.347903 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:12.347875    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:12.348972 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:12.348955    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:31.451949 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:31.451867    2549 generic.go:356] "Generic (PLEG): container finished" podID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerID="f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9" exitCode=0
Jul 28 13:24:31.452454 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:31.451945    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" event={"ID":"bdb1f775-25ca-439c-9e5c-660f7703e36a","Type":"ContainerDied","Data":"f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9"}
Jul 28 13:24:31.452516 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:31.452468    2549 scope.go:117] "RemoveContainer" containerID="f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9"
Jul 28 13:24:31.453662 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:31.453607    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:24:31.453766 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:31.453710    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:24:31.453766 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:31.453746    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:24:32.459558 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:32.459521    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" event={"ID":"bdb1f775-25ca-439c-9e5c-660f7703e36a","Type":"ContainerStarted","Data":"032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c"}
Jul 28 13:24:33.438711 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:33.438665    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:33.438983 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:33.438958    2549 patch_prober.go:28] interesting pod/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 28 13:24:33.439050 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:33.439029    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 28 13:24:34.439539 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:34.439503    2549 patch_prober.go:28] interesting pod/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 28 13:24:34.440060 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:34.439578    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 28 13:24:35.438864 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:35.438832    2549 patch_prober.go:28] interesting pod/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 28 13:24:35.439061 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:35.438893    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 28 13:24:36.442318 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:36.442286    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:36.442752 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:36.442558    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:24:36.443193 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:24:36.443177    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:25:08.854062 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:08.854038    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:25:08.866569 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:08.866544    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tctq_3a86f337-8566-420f-aab9-5d9329d8982c/ovn-acl-logging/0.log"
Jul 28 13:25:23.013266 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.013225    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"]
Jul 28 13:25:23.016010 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.015987    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.018790 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.018770    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-33eb65-openshift-default-dockercfg-zfffk\""
Jul 28 13:25:23.030788 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.030749    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"]
Jul 28 13:25:23.184162 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.184126    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-envoy\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.184324 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.184168    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-podinfo\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.184324 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.184262    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e64db9fa-8d65-4bae-bf49-d84166a77877-istiod-ca-cert\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.184324 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.184303    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-credential-socket\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.184437 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.184337    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-token\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.184437 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.184403    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-socket\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.184437 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.184428    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-data\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.184530 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.184453    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sptfm\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-kube-api-access-sptfm\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.184530 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.184474    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-certs\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.285988 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.285896    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e64db9fa-8d65-4bae-bf49-d84166a77877-istiod-ca-cert\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.285988 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.285930    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-credential-socket\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.285988 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.285950    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-token\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.285988 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.285987    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-socket\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286336 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286009    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-data\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286336 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286035    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sptfm\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-kube-api-access-sptfm\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286336 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286063    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-certs\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286336 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286150    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-envoy\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286336 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286185    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-podinfo\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286594 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286402    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-credential-socket\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286594 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286430    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-socket\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286594 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286560    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-data\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286594 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286577    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-certs\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.286932 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.286911    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e64db9fa-8d65-4bae-bf49-d84166a77877-istiod-ca-cert\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.288606 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.288584    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-envoy\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.288812 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.288794    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-podinfo\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.294814 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.294792    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-token\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.294927 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.294850    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sptfm\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-kube-api-access-sptfm\") pod \"e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") " pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.326601 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.326571    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:23.472408 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.472369    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"]
Jul 28 13:25:23.483119 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.483074    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:25:23.483227 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.483158    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:25:23.483227 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.483196    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:25:23.742928 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.742892    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" event={"ID":"e64db9fa-8d65-4bae-bf49-d84166a77877","Type":"ContainerStarted","Data":"0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890"}
Jul 28 13:25:23.743103 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.742955    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" event={"ID":"e64db9fa-8d65-4bae-bf49-d84166a77877","Type":"ContainerStarted","Data":"4391ef5397f322ba9f50086c1acb3dd65d4544c9ba62df951ef51a5ac35f6f72"}
Jul 28 13:25:23.764209 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:23.764156    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" podStartSLOduration=1.7641401289999998 podStartE2EDuration="1.764140129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:25:23.763576153 +0000 UTC m=+3315.627826024" watchObservedRunningTime="2026-07-28 13:25:23.764140129 +0000 UTC m=+3315.628389987"
Jul 28 13:25:24.326870 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:24.326840    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:24.332342 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:24.332316    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:24.748293 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:24.748216    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:24.749134 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:24.749115    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:44.863076 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:44.863042    2549 generic.go:356] "Generic (PLEG): container finished" podID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerID="0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890" exitCode=0
Jul 28 13:25:44.863611 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:44.863113    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" event={"ID":"e64db9fa-8d65-4bae-bf49-d84166a77877","Type":"ContainerDied","Data":"0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890"}
Jul 28 13:25:44.863715 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:44.863640    2549 scope.go:117] "RemoveContainer" containerID="0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890"
Jul 28 13:25:44.864815 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:44.864790    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:25:44.864889 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:44.864861    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:25:44.864933 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:44.864894    2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 28 13:25:45.871426 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:45.871387    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" event={"ID":"e64db9fa-8d65-4bae-bf49-d84166a77877","Type":"ContainerStarted","Data":"998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9"}
Jul 28 13:25:46.327639 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:46.327574    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:46.327821 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:46.327734    2549 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 28 13:25:46.327821 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:46.327785    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 28 13:25:47.327221 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:47.327184    2549 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 28 13:25:47.327721 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:47.327245    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 28 13:25:48.327892 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:48.327856    2549 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 28 13:25:48.328289 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:48.327917    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 28 13:25:49.327272 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:49.327237    2549 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 28 13:25:49.327450 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:49.327311    2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 28 13:25:50.332192 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:50.332158    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:50.332730 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:50.332574    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:25:50.333287 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:25:50.333271    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:26:02.983995 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:02.983961    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"]
Jul 28 13:26:02.984462 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:02.984250    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy" containerID="cri-o://998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9" gracePeriod=30
Jul 28 13:26:05.335147 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:05.335111    2549 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 28 13:26:05.335540 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:05.335179    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 13:26:08.244830 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.244806    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:26:08.305745 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.305653    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-token\") pod \"e64db9fa-8d65-4bae-bf49-d84166a77877\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") "
Jul 28 13:26:08.305745 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.305740    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-data\") pod \"e64db9fa-8d65-4bae-bf49-d84166a77877\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") "
Jul 28 13:26:08.306064 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.305784    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-socket\") pod \"e64db9fa-8d65-4bae-bf49-d84166a77877\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") "
Jul 28 13:26:08.306064 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.305834    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-podinfo\") pod \"e64db9fa-8d65-4bae-bf49-d84166a77877\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") "
Jul 28 13:26:08.306064 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.305915    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-envoy\") pod \"e64db9fa-8d65-4bae-bf49-d84166a77877\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") "
Jul 28 13:26:08.306064 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.305953    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sptfm\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-kube-api-access-sptfm\") pod \"e64db9fa-8d65-4bae-bf49-d84166a77877\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") "
Jul 28 13:26:08.306064 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.305978    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-certs\") pod \"e64db9fa-8d65-4bae-bf49-d84166a77877\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") "
Jul 28 13:26:08.306064 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.306010    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e64db9fa-8d65-4bae-bf49-d84166a77877-istiod-ca-cert\") pod \"e64db9fa-8d65-4bae-bf49-d84166a77877\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") "
Jul 28 13:26:08.306064 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.306036    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-credential-socket\") pod \"e64db9fa-8d65-4bae-bf49-d84166a77877\" (UID: \"e64db9fa-8d65-4bae-bf49-d84166a77877\") "
Jul 28 13:26:08.307043 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.306088    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e64db9fa-8d65-4bae-bf49-d84166a77877" (UID: "e64db9fa-8d65-4bae-bf49-d84166a77877"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:26:08.307043 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.306392    2549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-socket\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:08.307043 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.306441    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e64db9fa-8d65-4bae-bf49-d84166a77877" (UID: "e64db9fa-8d65-4bae-bf49-d84166a77877"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:26:08.307043 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.306603    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e64db9fa-8d65-4bae-bf49-d84166a77877" (UID: "e64db9fa-8d65-4bae-bf49-d84166a77877"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:26:08.307043 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.306603    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e64db9fa-8d65-4bae-bf49-d84166a77877-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e64db9fa-8d65-4bae-bf49-d84166a77877" (UID: "e64db9fa-8d65-4bae-bf49-d84166a77877"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:26:08.307043 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.306750    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-data" (OuterVolumeSpecName: "istio-data") pod "e64db9fa-8d65-4bae-bf49-d84166a77877" (UID: "e64db9fa-8d65-4bae-bf49-d84166a77877"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:26:08.308440 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.308411    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e64db9fa-8d65-4bae-bf49-d84166a77877" (UID: "e64db9fa-8d65-4bae-bf49-d84166a77877"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 13:26:08.308585 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.308565    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e64db9fa-8d65-4bae-bf49-d84166a77877" (UID: "e64db9fa-8d65-4bae-bf49-d84166a77877"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:26:08.308910 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.308888    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-token" (OuterVolumeSpecName: "istio-token") pod "e64db9fa-8d65-4bae-bf49-d84166a77877" (UID: "e64db9fa-8d65-4bae-bf49-d84166a77877"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:26:08.309030 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.308963    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-kube-api-access-sptfm" (OuterVolumeSpecName: "kube-api-access-sptfm") pod "e64db9fa-8d65-4bae-bf49-d84166a77877" (UID: "e64db9fa-8d65-4bae-bf49-d84166a77877"). InnerVolumeSpecName "kube-api-access-sptfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:26:08.407682 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.407651    2549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-envoy\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:08.407682 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.407681    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sptfm\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-kube-api-access-sptfm\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:08.407911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.407697    2549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-workload-certs\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:08.407911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.407710    2549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e64db9fa-8d65-4bae-bf49-d84166a77877-istiod-ca-cert\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:08.407911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.407718    2549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-credential-socket\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:08.407911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.407725    2549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-token\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:08.407911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.407733    2549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-data\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:08.407911 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:08.407740    2549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e64db9fa-8d65-4bae-bf49-d84166a77877-istio-podinfo\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:09.009935 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.009855    2549 generic.go:356] "Generic (PLEG): container finished" podID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerID="998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9" exitCode=0
Jul 28 13:26:09.009935 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.009903    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"
Jul 28 13:26:09.010117 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.009947    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" event={"ID":"e64db9fa-8d65-4bae-bf49-d84166a77877","Type":"ContainerDied","Data":"998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9"}
Jul 28 13:26:09.010117 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.010004    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7" event={"ID":"e64db9fa-8d65-4bae-bf49-d84166a77877","Type":"ContainerDied","Data":"4391ef5397f322ba9f50086c1acb3dd65d4544c9ba62df951ef51a5ac35f6f72"}
Jul 28 13:26:09.010117 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.010030    2549 scope.go:117] "RemoveContainer" containerID="998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9"
Jul 28 13:26:09.019521 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.019502    2549 scope.go:117] "RemoveContainer" containerID="0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890"
Jul 28 13:26:09.028505 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.028485    2549 scope.go:117] "RemoveContainer" containerID="998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9"
Jul 28 13:26:09.028811 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:26:09.028787    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9\": container with ID starting with 998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9 not found: ID does not exist" containerID="998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9"
Jul 28 13:26:09.028871 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.028821    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9"} err="failed to get container status \"998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9\": rpc error: code = NotFound desc = could not find container \"998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9\": container with ID starting with 998827ff8779fbd48f1e86a1e54a1e3a47e03866b76ac53b864c7d6e4a4c00d9 not found: ID does not exist"
Jul 28 13:26:09.028871 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.028839    2549 scope.go:117] "RemoveContainer" containerID="0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890"
Jul 28 13:26:09.029091 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:26:09.029076    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890\": container with ID starting with 0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890 not found: ID does not exist" containerID="0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890"
Jul 28 13:26:09.029146 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.029093    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890"} err="failed to get container status \"0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890\": rpc error: code = NotFound desc = could not find container \"0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890\": container with ID starting with 0732ef7d9cf251cfab21ed9fa1002cfa52db37912d75d202a7acaf391510c890 not found: ID does not exist"
Jul 28 13:26:09.033341 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.033319    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"]
Jul 28 13:26:09.039404 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:09.039379    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-33eb65-openshift-default-9fc9764b8-jlfd7"]
Jul 28 13:26:10.693071 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:10.693040    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" path="/var/lib/kubelet/pods/e64db9fa-8d65-4bae-bf49-d84166a77877/volumes"
Jul 28 13:26:22.544418 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:22.544375    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"]
Jul 28 13:26:22.545005 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:22.544688    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn" podUID="1390502d-0e9e-4dee-9840-1a154672c9ce" containerName="maas-api" containerID="cri-o://2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb" gracePeriod=30
Jul 28 13:26:22.619264 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:22.619227    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"]
Jul 28 13:26:22.619529 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:22.619501    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4" podUID="906297b7-e672-4b85-9d92-2146b1490975" containerName="payload-pre-processing" containerID="cri-o://a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d" gracePeriod=30
Jul 28 13:26:22.892832 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:22.892802    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:26:22.926865 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:22.926837    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:26:23.046148 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.046116    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxhj7\" (UniqueName: \"kubernetes.io/projected/1390502d-0e9e-4dee-9840-1a154672c9ce-kube-api-access-fxhj7\") pod \"1390502d-0e9e-4dee-9840-1a154672c9ce\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") "
Jul 28 13:26:23.046148 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.046148    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906297b7-e672-4b85-9d92-2146b1490975-plugins-config-volume\") pod \"906297b7-e672-4b85-9d92-2146b1490975\" (UID: \"906297b7-e672-4b85-9d92-2146b1490975\") "
Jul 28 13:26:23.046348 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.046200    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jgtn\" (UniqueName: \"kubernetes.io/projected/906297b7-e672-4b85-9d92-2146b1490975-kube-api-access-8jgtn\") pod \"906297b7-e672-4b85-9d92-2146b1490975\" (UID: \"906297b7-e672-4b85-9d92-2146b1490975\") "
Jul 28 13:26:23.046348 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.046236    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1390502d-0e9e-4dee-9840-1a154672c9ce-openshift-service-ca\") pod \"1390502d-0e9e-4dee-9840-1a154672c9ce\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") "
Jul 28 13:26:23.046348 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.046305    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1390502d-0e9e-4dee-9840-1a154672c9ce-maas-api-tls\") pod \"1390502d-0e9e-4dee-9840-1a154672c9ce\" (UID: \"1390502d-0e9e-4dee-9840-1a154672c9ce\") "
Jul 28 13:26:23.046643 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.046596    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/906297b7-e672-4b85-9d92-2146b1490975-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "906297b7-e672-4b85-9d92-2146b1490975" (UID: "906297b7-e672-4b85-9d92-2146b1490975"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:26:23.046750 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.046667    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1390502d-0e9e-4dee-9840-1a154672c9ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1390502d-0e9e-4dee-9840-1a154672c9ce" (UID: "1390502d-0e9e-4dee-9840-1a154672c9ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:26:23.046750 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.046700    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906297b7-e672-4b85-9d92-2146b1490975-plugins-config-volume\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:23.048441 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.048415    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906297b7-e672-4b85-9d92-2146b1490975-kube-api-access-8jgtn" (OuterVolumeSpecName: "kube-api-access-8jgtn") pod "906297b7-e672-4b85-9d92-2146b1490975" (UID: "906297b7-e672-4b85-9d92-2146b1490975"). InnerVolumeSpecName "kube-api-access-8jgtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:26:23.048441 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.048426    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1390502d-0e9e-4dee-9840-1a154672c9ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1390502d-0e9e-4dee-9840-1a154672c9ce" (UID: "1390502d-0e9e-4dee-9840-1a154672c9ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:26:23.048441 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.048437    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1390502d-0e9e-4dee-9840-1a154672c9ce-kube-api-access-fxhj7" (OuterVolumeSpecName: "kube-api-access-fxhj7") pod "1390502d-0e9e-4dee-9840-1a154672c9ce" (UID: "1390502d-0e9e-4dee-9840-1a154672c9ce"). InnerVolumeSpecName "kube-api-access-fxhj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:26:23.083597 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.083564    2549 generic.go:356] "Generic (PLEG): container finished" podID="1390502d-0e9e-4dee-9840-1a154672c9ce" containerID="2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb" exitCode=0
Jul 28 13:26:23.083762 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.083651    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"
Jul 28 13:26:23.083762 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.083650    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn" event={"ID":"1390502d-0e9e-4dee-9840-1a154672c9ce","Type":"ContainerDied","Data":"2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb"}
Jul 28 13:26:23.083762 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.083725    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn" event={"ID":"1390502d-0e9e-4dee-9840-1a154672c9ce","Type":"ContainerDied","Data":"2c162a2a2f4bfc316fba5c0eb6377813003bdc4442765b02847a166e9ecd6cbd"}
Jul 28 13:26:23.083762 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.083746    2549 scope.go:117] "RemoveContainer" containerID="2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb"
Jul 28 13:26:23.085178 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.085153    2549 generic.go:356] "Generic (PLEG): container finished" podID="906297b7-e672-4b85-9d92-2146b1490975" containerID="a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d" exitCode=0
Jul 28 13:26:23.085304 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.085197    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"
Jul 28 13:26:23.085304 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.085229    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4" event={"ID":"906297b7-e672-4b85-9d92-2146b1490975","Type":"ContainerDied","Data":"a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d"}
Jul 28 13:26:23.085304 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.085271    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4" event={"ID":"906297b7-e672-4b85-9d92-2146b1490975","Type":"ContainerDied","Data":"e54516e44890868cc27e8f199f862621e488712e96bf8802d2469a005e7b2c52"}
Jul 28 13:26:23.096473 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.096456    2549 scope.go:117] "RemoveContainer" containerID="2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb"
Jul 28 13:26:23.096869 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:26:23.096844    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb\": container with ID starting with 2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb not found: ID does not exist" containerID="2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb"
Jul 28 13:26:23.096981 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.096879    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb"} err="failed to get container status \"2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb\": rpc error: code = NotFound desc = could not find container \"2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb\": container with ID starting with 2abf084c0caeb79eb95de8bc2b0f496aaa76555eca72fc4d42a80bd3b94bfdeb not found: ID does not exist"
Jul 28 13:26:23.096981 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.096904    2549 scope.go:117] "RemoveContainer" containerID="a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d"
Jul 28 13:26:23.105461 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.105444    2549 scope.go:117] "RemoveContainer" containerID="a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d"
Jul 28 13:26:23.105741 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:26:23.105723    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d\": container with ID starting with a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d not found: ID does not exist" containerID="a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d"
Jul 28 13:26:23.105796 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.105748    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d"} err="failed to get container status \"a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d\": rpc error: code = NotFound desc = could not find container \"a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d\": container with ID starting with a9ff8045d6f11b9a35e9e8afe75763b49cdfad1cf8ce88d9f4e886e4fb3ac30d not found: ID does not exist"
Jul 28 13:26:23.111449 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.111427    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"]
Jul 28 13:26:23.114874 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.114851    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8fb56b-7c6b64fdb6-ptgmn"]
Jul 28 13:26:23.125594 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.125574    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"]
Jul 28 13:26:23.129059 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.129039    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8fb56b-57685bb6c6-nm9k4"]
Jul 28 13:26:23.147195 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.147169    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1390502d-0e9e-4dee-9840-1a154672c9ce-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:23.147195 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.147193    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1390502d-0e9e-4dee-9840-1a154672c9ce-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:23.147324 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.147203    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxhj7\" (UniqueName: \"kubernetes.io/projected/1390502d-0e9e-4dee-9840-1a154672c9ce-kube-api-access-fxhj7\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:23.147324 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:23.147212    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jgtn\" (UniqueName: \"kubernetes.io/projected/906297b7-e672-4b85-9d92-2146b1490975-kube-api-access-8jgtn\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:26:24.692057 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:24.692021    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1390502d-0e9e-4dee-9840-1a154672c9ce" path="/var/lib/kubelet/pods/1390502d-0e9e-4dee-9840-1a154672c9ce/volumes"
Jul 28 13:26:24.692461 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:26:24.692383    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="906297b7-e672-4b85-9d92-2146b1490975" path="/var/lib/kubelet/pods/906297b7-e672-4b85-9d92-2146b1490975/volumes"
Jul 28 13:27:45.615830 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:45.615792    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"]
Jul 28 13:27:45.616421 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:45.616017    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerName="istio-proxy" containerID="cri-o://032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c" gracePeriod=30
Jul 28 13:27:50.881915 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.881841    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:27:50.982676 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982611    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-socket\") pod \"bdb1f775-25ca-439c-9e5c-660f7703e36a\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") "
Jul 28 13:27:50.982676 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982670    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-envoy\") pod \"bdb1f775-25ca-439c-9e5c-660f7703e36a\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") "
Jul 28 13:27:50.983009 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982693    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zd6v\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-kube-api-access-7zd6v\") pod \"bdb1f775-25ca-439c-9e5c-660f7703e36a\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") "
Jul 28 13:27:50.983009 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982741    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-podinfo\") pod \"bdb1f775-25ca-439c-9e5c-660f7703e36a\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") "
Jul 28 13:27:50.983009 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982756    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-certs\") pod \"bdb1f775-25ca-439c-9e5c-660f7703e36a\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") "
Jul 28 13:27:50.983009 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982773    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-token\") pod \"bdb1f775-25ca-439c-9e5c-660f7703e36a\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") "
Jul 28 13:27:50.983009 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982849    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb1f775-25ca-439c-9e5c-660f7703e36a-istiod-ca-cert\") pod \"bdb1f775-25ca-439c-9e5c-660f7703e36a\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") "
Jul 28 13:27:50.983009 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982891    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-data\") pod \"bdb1f775-25ca-439c-9e5c-660f7703e36a\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") "
Jul 28 13:27:50.983009 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982925    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-credential-socket\") pod \"bdb1f775-25ca-439c-9e5c-660f7703e36a\" (UID: \"bdb1f775-25ca-439c-9e5c-660f7703e36a\") "
Jul 28 13:27:50.983009 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.982971    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bdb1f775-25ca-439c-9e5c-660f7703e36a" (UID: "bdb1f775-25ca-439c-9e5c-660f7703e36a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:27:50.983443 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.983259    2549 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-socket\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:50.983443 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.983280    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdb1f775-25ca-439c-9e5c-660f7703e36a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bdb1f775-25ca-439c-9e5c-660f7703e36a" (UID: "bdb1f775-25ca-439c-9e5c-660f7703e36a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:27:50.983443 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.983320    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bdb1f775-25ca-439c-9e5c-660f7703e36a" (UID: "bdb1f775-25ca-439c-9e5c-660f7703e36a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:27:50.983822 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.983643    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bdb1f775-25ca-439c-9e5c-660f7703e36a" (UID: "bdb1f775-25ca-439c-9e5c-660f7703e36a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:27:50.984249 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.984224    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-data" (OuterVolumeSpecName: "istio-data") pod "bdb1f775-25ca-439c-9e5c-660f7703e36a" (UID: "bdb1f775-25ca-439c-9e5c-660f7703e36a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:27:50.985124 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.985103    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bdb1f775-25ca-439c-9e5c-660f7703e36a" (UID: "bdb1f775-25ca-439c-9e5c-660f7703e36a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 13:27:50.985232 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.985190    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bdb1f775-25ca-439c-9e5c-660f7703e36a" (UID: "bdb1f775-25ca-439c-9e5c-660f7703e36a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 13:27:50.985539 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.985514    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-token" (OuterVolumeSpecName: "istio-token") pod "bdb1f775-25ca-439c-9e5c-660f7703e36a" (UID: "bdb1f775-25ca-439c-9e5c-660f7703e36a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:27:50.985539 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:50.985532    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-kube-api-access-7zd6v" (OuterVolumeSpecName: "kube-api-access-7zd6v") pod "bdb1f775-25ca-439c-9e5c-660f7703e36a" (UID: "bdb1f775-25ca-439c-9e5c-660f7703e36a"). InnerVolumeSpecName "kube-api-access-7zd6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:27:51.084157 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.084116    2549 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-podinfo\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:51.084157 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.084153    2549 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-workload-certs\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:51.084157 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.084162    2549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-token\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:51.084378 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.084169    2549 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb1f775-25ca-439c-9e5c-660f7703e36a-istiod-ca-cert\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:51.084378 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.084176    2549 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-data\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:51.084378 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.084184    2549 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-credential-socket\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:51.084378 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.084193    2549 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb1f775-25ca-439c-9e5c-660f7703e36a-istio-envoy\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:51.084378 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.084201    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zd6v\" (UniqueName: \"kubernetes.io/projected/bdb1f775-25ca-439c-9e5c-660f7703e36a-kube-api-access-7zd6v\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:51.551589 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.551554    2549 generic.go:356] "Generic (PLEG): container finished" podID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerID="032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c" exitCode=0
Jul 28 13:27:51.551849 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.551616    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"
Jul 28 13:27:51.551849 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.551663    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" event={"ID":"bdb1f775-25ca-439c-9e5c-660f7703e36a","Type":"ContainerDied","Data":"032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c"}
Jul 28 13:27:51.551849 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.551747    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d" event={"ID":"bdb1f775-25ca-439c-9e5c-660f7703e36a","Type":"ContainerDied","Data":"d8c2a91dccfbc2e0a0f3359208520cc76a006fe7443d23de86ed88de6c4060ca"}
Jul 28 13:27:51.551849 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.551771    2549 scope.go:117] "RemoveContainer" containerID="032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c"
Jul 28 13:27:51.561604 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.561589    2549 scope.go:117] "RemoveContainer" containerID="f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9"
Jul 28 13:27:51.570457 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.570437    2549 scope.go:117] "RemoveContainer" containerID="032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c"
Jul 28 13:27:51.570750 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:27:51.570729    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c\": container with ID starting with 032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c not found: ID does not exist" containerID="032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c"
Jul 28 13:27:51.570821 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.570758    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c"} err="failed to get container status \"032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c\": rpc error: code = NotFound desc = could not find container \"032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c\": container with ID starting with 032f37e7f031bd132e0f13226aeb6c3372b422f31510b8a520e73b1120ece27c not found: ID does not exist"
Jul 28 13:27:51.570821 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.570775    2549 scope.go:117] "RemoveContainer" containerID="f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9"
Jul 28 13:27:51.571041 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:27:51.571024    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9\": container with ID starting with f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9 not found: ID does not exist" containerID="f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9"
Jul 28 13:27:51.571083 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.571051    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9"} err="failed to get container status \"f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9\": rpc error: code = NotFound desc = could not find container \"f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9\": container with ID starting with f1e066f92106ae45d6e731508c7f9db1baaae48d51fb3447651e518d539145b9 not found: ID does not exist"
Jul 28 13:27:51.574914 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.574891    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"]
Jul 28 13:27:51.578922 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:51.578902    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-06a4d2-openshift-default-d4d469978-wjg4d"]
Jul 28 13:27:52.693172 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:52.693139    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" path="/var/lib/kubelet/pods/bdb1f775-25ca-439c-9e5c-660f7703e36a/volumes"
Jul 28 13:27:56.631876 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:56.631841    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"]
Jul 28 13:27:56.632264 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:56.632136    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg" podUID="079edf1c-9dc1-44e0-accb-c98b172d4bf6" containerName="maas-api" containerID="cri-o://2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f" gracePeriod=30
Jul 28 13:27:56.899464 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:56.899390    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:27:56.939693 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:56.939661    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079edf1c-9dc1-44e0-accb-c98b172d4bf6-maas-api-tls\") pod \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") "
Jul 28 13:27:56.939858 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:56.939762    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079edf1c-9dc1-44e0-accb-c98b172d4bf6-openshift-service-ca\") pod \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") "
Jul 28 13:27:56.939858 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:56.939815    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8gnn\" (UniqueName: \"kubernetes.io/projected/079edf1c-9dc1-44e0-accb-c98b172d4bf6-kube-api-access-g8gnn\") pod \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\" (UID: \"079edf1c-9dc1-44e0-accb-c98b172d4bf6\") "
Jul 28 13:27:56.940129 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:56.940107    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/079edf1c-9dc1-44e0-accb-c98b172d4bf6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "079edf1c-9dc1-44e0-accb-c98b172d4bf6" (UID: "079edf1c-9dc1-44e0-accb-c98b172d4bf6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:27:56.942135 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:56.942079    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/079edf1c-9dc1-44e0-accb-c98b172d4bf6-kube-api-access-g8gnn" (OuterVolumeSpecName: "kube-api-access-g8gnn") pod "079edf1c-9dc1-44e0-accb-c98b172d4bf6" (UID: "079edf1c-9dc1-44e0-accb-c98b172d4bf6"). InnerVolumeSpecName "kube-api-access-g8gnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:27:56.942235 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:56.942147    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/079edf1c-9dc1-44e0-accb-c98b172d4bf6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "079edf1c-9dc1-44e0-accb-c98b172d4bf6" (UID: "079edf1c-9dc1-44e0-accb-c98b172d4bf6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:27:57.041095 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.041061    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079edf1c-9dc1-44e0-accb-c98b172d4bf6-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:57.041095 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.041090    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079edf1c-9dc1-44e0-accb-c98b172d4bf6-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:57.041095 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.041101    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8gnn\" (UniqueName: \"kubernetes.io/projected/079edf1c-9dc1-44e0-accb-c98b172d4bf6-kube-api-access-g8gnn\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:27:57.591996 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.591958    2549 generic.go:356] "Generic (PLEG): container finished" podID="079edf1c-9dc1-44e0-accb-c98b172d4bf6" containerID="2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f" exitCode=0
Jul 28 13:27:57.592186 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.591996    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg" event={"ID":"079edf1c-9dc1-44e0-accb-c98b172d4bf6","Type":"ContainerDied","Data":"2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f"}
Jul 28 13:27:57.592186 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.592070    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"
Jul 28 13:27:57.592186 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.592096    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg" event={"ID":"079edf1c-9dc1-44e0-accb-c98b172d4bf6","Type":"ContainerDied","Data":"cd16fece80dd5e2a922ab741df031a1240e6880a293fc2fc252b4a787062fad4"}
Jul 28 13:27:57.592186 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.592128    2549 scope.go:117] "RemoveContainer" containerID="2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f"
Jul 28 13:27:57.602045 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.602023    2549 scope.go:117] "RemoveContainer" containerID="2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f"
Jul 28 13:27:57.602293 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:27:57.602273    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f\": container with ID starting with 2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f not found: ID does not exist" containerID="2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f"
Jul 28 13:27:57.602341 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.602303    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f"} err="failed to get container status \"2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f\": rpc error: code = NotFound desc = could not find container \"2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f\": container with ID starting with 2b4ce5ae50a54254b82728e4d5c5d4dd2df376c24d46376d5b550d46441acc1f not found: ID does not exist"
Jul 28 13:27:57.615940 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.615915    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"]
Jul 28 13:27:57.619590 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:57.619568    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67d68f-594cd7f6cd-p9pzg"]
Jul 28 13:27:58.692158 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:27:58.692124    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="079edf1c-9dc1-44e0-accb-c98b172d4bf6" path="/var/lib/kubelet/pods/079edf1c-9dc1-44e0-accb-c98b172d4bf6/volumes"
Jul 28 13:28:18.582062 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:18.582031    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"]
Jul 28 13:28:18.582528 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:18.582227    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr" podUID="2e31ec89-4055-4b25-a0c6-3ed80acaa14f" containerName="maas-api" containerID="cri-o://3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93" gracePeriod=30
Jul 28 13:28:18.625536 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:18.625505    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"]
Jul 28 13:28:18.625796 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:18.625773    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2" podUID="d4a23b0a-3b17-48f2-a25e-0944e34cc6fe" containerName="payload-processing" containerID="cri-o://9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1" gracePeriod=30
Jul 28 13:28:18.882543 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:18.882508    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:28:18.919613 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:18.919583    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:28:19.044521 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.044485    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-openshift-service-ca\") pod \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") "
Jul 28 13:28:19.044746 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.044536    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-maas-api-tls\") pod \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") "
Jul 28 13:28:19.044746 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.044573    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhxqb\" (UniqueName: \"kubernetes.io/projected/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-kube-api-access-mhxqb\") pod \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\" (UID: \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\") "
Jul 28 13:28:19.044746 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.044609    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-plugins-config-volume\") pod \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\" (UID: \"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe\") "
Jul 28 13:28:19.044746 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.044680    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgf9n\" (UniqueName: \"kubernetes.io/projected/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-kube-api-access-pgf9n\") pod \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\" (UID: \"2e31ec89-4055-4b25-a0c6-3ed80acaa14f\") "
Jul 28 13:28:19.045022 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.044992    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2e31ec89-4055-4b25-a0c6-3ed80acaa14f" (UID: "2e31ec89-4055-4b25-a0c6-3ed80acaa14f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:28:19.045077 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.045061    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4a23b0a-3b17-48f2-a25e-0944e34cc6fe" (UID: "d4a23b0a-3b17-48f2-a25e-0944e34cc6fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 13:28:19.046937 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.046912    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-kube-api-access-mhxqb" (OuterVolumeSpecName: "kube-api-access-mhxqb") pod "d4a23b0a-3b17-48f2-a25e-0944e34cc6fe" (UID: "d4a23b0a-3b17-48f2-a25e-0944e34cc6fe"). InnerVolumeSpecName "kube-api-access-mhxqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:28:19.047049 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.046931    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2e31ec89-4055-4b25-a0c6-3ed80acaa14f" (UID: "2e31ec89-4055-4b25-a0c6-3ed80acaa14f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 13:28:19.047049 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.046934    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-kube-api-access-pgf9n" (OuterVolumeSpecName: "kube-api-access-pgf9n") pod "2e31ec89-4055-4b25-a0c6-3ed80acaa14f" (UID: "2e31ec89-4055-4b25-a0c6-3ed80acaa14f"). InnerVolumeSpecName "kube-api-access-pgf9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 13:28:19.145887 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.145811    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-openshift-service-ca\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:28:19.145887 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.145837    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-maas-api-tls\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:28:19.145887 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.145846    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhxqb\" (UniqueName: \"kubernetes.io/projected/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-kube-api-access-mhxqb\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:28:19.145887 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.145867    2549 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe-plugins-config-volume\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:28:19.145887 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.145875    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgf9n\" (UniqueName: \"kubernetes.io/projected/2e31ec89-4055-4b25-a0c6-3ed80acaa14f-kube-api-access-pgf9n\") on node \"ip-10-0-132-90.ec2.internal\" DevicePath \"\""
Jul 28 13:28:19.708043 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.708014    2549 generic.go:356] "Generic (PLEG): container finished" podID="2e31ec89-4055-4b25-a0c6-3ed80acaa14f" containerID="3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93" exitCode=0
Jul 28 13:28:19.708506 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.708061    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"
Jul 28 13:28:19.708506 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.708102    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr" event={"ID":"2e31ec89-4055-4b25-a0c6-3ed80acaa14f","Type":"ContainerDied","Data":"3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93"}
Jul 28 13:28:19.708506 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.708152    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr" event={"ID":"2e31ec89-4055-4b25-a0c6-3ed80acaa14f","Type":"ContainerDied","Data":"67c18c3d3d662b7b4140c35cc93a9d41a378c5591e73ab3948324d1693f2cd3c"}
Jul 28 13:28:19.708506 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.708174    2549 scope.go:117] "RemoveContainer" containerID="3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93"
Jul 28 13:28:19.709608 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.709585    2549 generic.go:356] "Generic (PLEG): container finished" podID="d4a23b0a-3b17-48f2-a25e-0944e34cc6fe" containerID="9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1" exitCode=0
Jul 28 13:28:19.709733 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.709650    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"
Jul 28 13:28:19.709794 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.709662    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2" event={"ID":"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe","Type":"ContainerDied","Data":"9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1"}
Jul 28 13:28:19.709894 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.709880    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2" event={"ID":"d4a23b0a-3b17-48f2-a25e-0944e34cc6fe","Type":"ContainerDied","Data":"d856074b9f07c8083068d9b0732e98b9107524045e9f734b5ba3321f7c4a7036"}
Jul 28 13:28:19.717259 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.717236    2549 scope.go:117] "RemoveContainer" containerID="3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93"
Jul 28 13:28:19.717543 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:28:19.717525    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93\": container with ID starting with 3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93 not found: ID does not exist" containerID="3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93"
Jul 28 13:28:19.717647 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.717551    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93"} err="failed to get container status \"3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93\": rpc error: code = NotFound desc = could not find container \"3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93\": container with ID starting with 3c5bc0b9ed0066f5b2d78f81c831fd8a21a6d0842c83ff77cb53f076c5968a93 not found: ID does not exist"
Jul 28 13:28:19.717647 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.717568    2549 scope.go:117] "RemoveContainer" containerID="9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1"
Jul 28 13:28:19.725938 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.725920    2549 scope.go:117] "RemoveContainer" containerID="9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1"
Jul 28 13:28:19.726214 ip-10-0-132-90 kubenswrapper[2549]: E0728 13:28:19.726187    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1\": container with ID starting with 9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1 not found: ID does not exist" containerID="9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1"
Jul 28 13:28:19.726301 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.726216    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1"} err="failed to get container status \"9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1\": rpc error: code = NotFound desc = could not find container \"9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1\": container with ID starting with 9869a717409f7c81abae38396968c9a21b43df7e760871c6a23a0fdb0dbfb5f1 not found: ID does not exist"
Jul 28 13:28:19.734615 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.734593    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"]
Jul 28 13:28:19.738447 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.738422    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-827464-598f8f8cf8-m9gb2"]
Jul 28 13:28:19.749234 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.749211    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"]
Jul 28 13:28:19.753873 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:19.753849    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-827464-646b444f77-lsjxr"]
Jul 28 13:28:20.692249 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:20.692208    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e31ec89-4055-4b25-a0c6-3ed80acaa14f" path="/var/lib/kubelet/pods/2e31ec89-4055-4b25-a0c6-3ed80acaa14f/volumes"
Jul 28 13:28:20.692781 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:20.692759    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4a23b0a-3b17-48f2-a25e-0944e34cc6fe" path="/var/lib/kubelet/pods/d4a23b0a-3b17-48f2-a25e-0944e34cc6fe/volumes"
Jul 28 13:28:30.599534 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:30.599438    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74d7594bf4-xc4ht_ea856dd5-8ac0-476b-87bf-5abb684f3cdc/authorino/0.log"
Jul 28 13:28:36.855473 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:36.855438    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-flt4v_033608ea-141b-49ce-a3c7-b9b5e2c43924/manager/0.log"
Jul 28 13:28:36.982221 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:36.982185    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rx4f2_288e06ff-932a-4134-80a1-3fe602c1e6dd/manager/0.log"
Jul 28 13:28:37.115190 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:37.115104    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65dd8bc54b-vnqz4_a55f2b36-a833-46cf-8c4a-de402175e5a1/manager/0.log"
Jul 28 13:28:37.235396 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:37.235368    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-f26sg_0f12ed3b-d534-49ed-996b-d079bfaa4d0d/server/0.log"
Jul 28 13:28:37.364174 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:37.364139    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cbdrp_9c6e1025-9697-41d3-a393-b3d8e84b5098/manager/0.log"
Jul 28 13:28:37.498764 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:37.498689    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7974886bdb-2rkn7_7b7043db-c8ad-4133-9d65-69ea83a34fe5/copy-manifests/0.log"
Jul 28 13:28:37.505845 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:37.505819    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7974886bdb-2rkn7_7b7043db-c8ad-4133-9d65-69ea83a34fe5/manager/0.log"
Jul 28 13:28:38.218718 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:38.218681    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8649546779-m2tj8_275f404d-e226-45f8-a411-d86b8b6d2963/maas-api/0.log"
Jul 28 13:28:38.361855 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:38.361821    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754045-gvzx5_3f3b9ade-1fa4-412f-b9f7-6a6f838e87f9/cleanup/0.log"
Jul 28 13:28:38.738865 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:38.738836    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-xffsd_6001921c-2d0b-47db-8c64-90f64c49ed8b/postgres/0.log"
Jul 28 13:28:40.193104 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:40.193073    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74d7594bf4-xc4ht_ea856dd5-8ac0-476b-87bf-5abb684f3cdc/authorino/0.log"
Jul 28 13:28:40.324557 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:40.324521    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dmrxr_0d3b7849-db09-41a2-8311-32cad5df1376/manager/0.log"
Jul 28 13:28:40.733974 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:40.733938    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-kp4jv_aa2edebf-b387-42de-9b5a-a243fb3ece48/registry-server/0.log"
Jul 28 13:28:40.897489 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:40.897459    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hpn2c_9899b67c-74bd-41e5-bd14-570f79d2204f/manager/0.log"
Jul 28 13:28:41.200820 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:41.200787    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2c98v_4f65d3d2-81e7-4ea9-98bd-e0fa933637c0/manager/0.log"
Jul 28 13:28:41.606564 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:41.606536    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29_bdb1d50f-0db7-4127-aa3a-65dad7bb40af/istio-proxy/0.log"
Jul 28 13:28:41.909067 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:41.908977    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b958f484-9pbr5_6187130a-311c-4bf7-88e1-18caab96d7e7/kube-auth-proxy/0.log"
Jul 28 13:28:42.215613 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:42.215528    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j4qxs_db993d21-5f17-4559-94b3-075a8a6acac2/istio-proxy/1.log"
Jul 28 13:28:42.357382 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:42.357351    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-r4f55_a7bfe86a-b217-4873-88b7-052fd8b09ad2/payload-pre-processing/0.log"
Jul 28 13:28:43.289203 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:43.289162    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2rfmw_d3b00dec-0ac2-4cec-8f1e-478cf0497327/main/0.log"
Jul 28 13:28:53.375305 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:53.375262    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sctjj_b4434036-47bd-45c9-b08b-12ab9f5f84c6/global-pull-secret-syncer/0.log"
Jul 28 13:28:53.555762 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:53.555716    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z7jj4_cee404ae-196d-4765-8dad-73a8c1da3793/konnectivity-agent/0.log"
Jul 28 13:28:53.606943 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:53.606907    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-90.ec2.internal_9415a092b6cee87f2b0f814e0eb8bce0/haproxy/0.log"
Jul 28 13:28:59.720025 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:59.719975    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74d7594bf4-xc4ht_ea856dd5-8ac0-476b-87bf-5abb684f3cdc/authorino/0.log"
Jul 28 13:28:59.748859 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:59.748834    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dmrxr_0d3b7849-db09-41a2-8311-32cad5df1376/manager/0.log"
Jul 28 13:28:59.884462 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:28:59.884434    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-kp4jv_aa2edebf-b387-42de-9b5a-a243fb3ece48/registry-server/0.log"
Jul 28 13:29:00.137930 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:00.137894    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hpn2c_9899b67c-74bd-41e5-bd14-570f79d2204f/manager/0.log"
Jul 28 13:29:00.285197 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:00.285161    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2c98v_4f65d3d2-81e7-4ea9-98bd-e0fa933637c0/manager/0.log"
Jul 28 13:29:02.006346 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.006307    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f690b07a-3ac7-44a1-b208-f004c7220c9e/alertmanager/0.log"
Jul 28 13:29:02.028146 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.028100    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f690b07a-3ac7-44a1-b208-f004c7220c9e/config-reloader/0.log"
Jul 28 13:29:02.052215 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.052198    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f690b07a-3ac7-44a1-b208-f004c7220c9e/kube-rbac-proxy-web/0.log"
Jul 28 13:29:02.074997 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.074978    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f690b07a-3ac7-44a1-b208-f004c7220c9e/kube-rbac-proxy/0.log"
Jul 28 13:29:02.097749 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.097730    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f690b07a-3ac7-44a1-b208-f004c7220c9e/kube-rbac-proxy-metric/0.log"
Jul 28 13:29:02.120849 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.120828    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f690b07a-3ac7-44a1-b208-f004c7220c9e/prom-label-proxy/0.log"
Jul 28 13:29:02.142842 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.142821    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f690b07a-3ac7-44a1-b208-f004c7220c9e/init-config-reloader/0.log"
Jul 28 13:29:02.225681 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.225652    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-t2dw4_7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f/kube-state-metrics/0.log"
Jul 28 13:29:02.242467 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.242444    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-t2dw4_7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f/kube-rbac-proxy-main/0.log"
Jul 28 13:29:02.264723 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.264655    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-t2dw4_7c18d8b7-57cf-4d3a-a20a-7b3d02b3e60f/kube-rbac-proxy-self/0.log"
Jul 28 13:29:02.325286 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.325260    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-5wzr7_a676e8b0-e920-43c5-96a7-64d8f56cc4cb/monitoring-plugin/0.log"
Jul 28 13:29:02.446415 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.446388    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c2p9n_76761059-1f76-43f6-8e6f-36ef294eba40/node-exporter/0.log"
Jul 28 13:29:02.467366 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.467338    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c2p9n_76761059-1f76-43f6-8e6f-36ef294eba40/kube-rbac-proxy/0.log"
Jul 28 13:29:02.511828 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.511806    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c2p9n_76761059-1f76-43f6-8e6f-36ef294eba40/init-textfile/0.log"
Jul 28 13:29:02.671810 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.671781    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-hrfss_c38306ff-de52-4ef3-932f-ac3b727114c3/kube-rbac-proxy-main/0.log"
Jul 28 13:29:02.692598 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.692579    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-hrfss_c38306ff-de52-4ef3-932f-ac3b727114c3/kube-rbac-proxy-self/0.log"
Jul 28 13:29:02.713603 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:02.713575    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-hrfss_c38306ff-de52-4ef3-932f-ac3b727114c3/openshift-state-metrics/0.log"
Jul 28 13:29:03.942708 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.942673    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"]
Jul 28 13:29:03.943138 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943123    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906297b7-e672-4b85-9d92-2146b1490975" containerName="payload-pre-processing"
Jul 28 13:29:03.943183 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943141    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="906297b7-e672-4b85-9d92-2146b1490975" containerName="payload-pre-processing"
Jul 28 13:29:03.943183 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943154    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1390502d-0e9e-4dee-9840-1a154672c9ce" containerName="maas-api"
Jul 28 13:29:03.943183 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943159    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="1390502d-0e9e-4dee-9840-1a154672c9ce" containerName="maas-api"
Jul 28 13:29:03.943183 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943166    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerName="istio-proxy"
Jul 28 13:29:03.943183 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943172    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerName="istio-proxy"
Jul 28 13:29:03.943183 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943182    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943188    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943199    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerName="istio-proxy"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943204    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerName="istio-proxy"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943216    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="079edf1c-9dc1-44e0-accb-c98b172d4bf6" containerName="maas-api"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943221    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="079edf1c-9dc1-44e0-accb-c98b172d4bf6" containerName="maas-api"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943226    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e31ec89-4055-4b25-a0c6-3ed80acaa14f" containerName="maas-api"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943231    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e31ec89-4055-4b25-a0c6-3ed80acaa14f" containerName="maas-api"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943243    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a23b0a-3b17-48f2-a25e-0944e34cc6fe" containerName="payload-processing"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943248    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a23b0a-3b17-48f2-a25e-0944e34cc6fe" containerName="payload-processing"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943303    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="906297b7-e672-4b85-9d92-2146b1490975" containerName="payload-pre-processing"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943311    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="1390502d-0e9e-4dee-9840-1a154672c9ce" containerName="maas-api"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943316    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdb1f775-25ca-439c-9e5c-660f7703e36a" containerName="istio-proxy"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943323    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943329    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e64db9fa-8d65-4bae-bf49-d84166a77877" containerName="istio-proxy"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943335    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a23b0a-3b17-48f2-a25e-0944e34cc6fe" containerName="payload-processing"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943340    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e31ec89-4055-4b25-a0c6-3ed80acaa14f" containerName="maas-api"
Jul 28 13:29:03.943400 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.943345    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="079edf1c-9dc1-44e0-accb-c98b172d4bf6" containerName="maas-api"
Jul 28 13:29:03.946747 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.946729    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:03.949606 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.949581    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5hhgv\"/\"default-dockercfg-tmkdf\""
Jul 28 13:29:03.949761 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.949581    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5hhgv\"/\"openshift-service-ca.crt\""
Jul 28 13:29:03.950959 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.950939    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5hhgv\"/\"kube-root-ca.crt\""
Jul 28 13:29:03.956334 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:03.956314    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"]
Jul 28 13:29:04.052093 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.052054    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-lib-modules\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.052258 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.052129    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-podres\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.052258 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.052174    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pv52\" (UniqueName: \"kubernetes.io/projected/fe766a19-f1ea-43f7-97af-ddde24d14f71-kube-api-access-5pv52\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.052258 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.052216    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-sys\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.052258 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.052255    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-proc\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.152916 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.152878    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-proc\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.153080 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.152934    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-lib-modules\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.153080 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.152973    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-podres\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.153080 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.152998    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pv52\" (UniqueName: \"kubernetes.io/projected/fe766a19-f1ea-43f7-97af-ddde24d14f71-kube-api-access-5pv52\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.153080 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.153014    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-proc\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.153080 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.153028    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-sys\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.153080 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.153079    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-sys\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.153341 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.153132    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-lib-modules\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.153341 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.153157    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fe766a19-f1ea-43f7-97af-ddde24d14f71-podres\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.162029 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.162008    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pv52\" (UniqueName: \"kubernetes.io/projected/fe766a19-f1ea-43f7-97af-ddde24d14f71-kube-api-access-5pv52\") pod \"perf-node-gather-daemonset-bwvzw\" (UID: \"fe766a19-f1ea-43f7-97af-ddde24d14f71\") " pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.257907 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.257827    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.390643 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.390597    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"]
Jul 28 13:29:04.392860 ip-10-0-132-90 kubenswrapper[2549]: W0728 13:29:04.392830    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfe766a19_f1ea_43f7_97af_ddde24d14f71.slice/crio-b3b3a9f2f460871c1f59752d5b96e903776e65553c98a653913e4708d0e042e6 WatchSource:0}: Error finding container b3b3a9f2f460871c1f59752d5b96e903776e65553c98a653913e4708d0e042e6: Status 404 returned error can't find the container with id b3b3a9f2f460871c1f59752d5b96e903776e65553c98a653913e4708d0e042e6
Jul 28 13:29:04.937053 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.937020    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw" event={"ID":"fe766a19-f1ea-43f7-97af-ddde24d14f71","Type":"ContainerStarted","Data":"b19fa6a8c05ce3baa0deedcf345d58c88339b36964e86d3c9cdf06acae329f06"}
Jul 28 13:29:04.937242 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.937076    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw" event={"ID":"fe766a19-f1ea-43f7-97af-ddde24d14f71","Type":"ContainerStarted","Data":"b3b3a9f2f460871c1f59752d5b96e903776e65553c98a653913e4708d0e042e6"}
Jul 28 13:29:04.937242 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.937107    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:04.952993 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:04.952940    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw" podStartSLOduration=1.952923822 podStartE2EDuration="1.952923822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:29:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:29:04.952137681 +0000 UTC m=+3536.816387540" watchObservedRunningTime="2026-07-28 13:29:04.952923822 +0000 UTC m=+3536.817173680"
Jul 28 13:29:07.224229 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:07.224199    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qc9cp_8168db1d-be3a-4555-9a2d-0876f4a02c1a/dns/0.log"
Jul 28 13:29:07.252694 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:07.252665    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qc9cp_8168db1d-be3a-4555-9a2d-0876f4a02c1a/kube-rbac-proxy/0.log"
Jul 28 13:29:07.326288 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:07.326247    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bhst2_d917c83e-d627-4099-ba45-58674437b5a8/dns-node-resolver/0.log"
Jul 28 13:29:07.960825 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:07.960789    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-bc8d7748d-6d7pj_2924e840-8e0a-4f46-b95a-5c5673c022c8/registry/0.log"
Jul 28 13:29:08.011262 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:08.011231    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qbgl6_f1b592e4-fbbf-4fe8-aa78-c10e97343d81/node-ca/0.log"
Jul 28 13:29:09.009258 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:09.009227    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29_bdb1d50f-0db7-4127-aa3a-65dad7bb40af/istio-proxy/0.log"
Jul 28 13:29:09.298490 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:09.298395    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b958f484-9pbr5_6187130a-311c-4bf7-88e1-18caab96d7e7/kube-auth-proxy/0.log"
Jul 28 13:29:09.357248 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:09.357211    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j4qxs_db993d21-5f17-4559-94b3-075a8a6acac2/istio-proxy/0.log"
Jul 28 13:29:09.410996 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:09.410963    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j4qxs_db993d21-5f17-4559-94b3-075a8a6acac2/istio-proxy/1.log"
Jul 28 13:29:09.451064 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:09.451022    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-r4f55_a7bfe86a-b217-4873-88b7-052fd8b09ad2/payload-pre-processing/0.log"
Jul 28 13:29:10.347427 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:10.347400    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hgnwd_12e4bb44-c3f8-48b5-bc8c-a43380db0819/serve-healthcheck-canary/0.log"
Jul 28 13:29:10.901939 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:10.901908    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j9sxv_9d4f01c5-13ee-42a1-9969-28b68e374307/kube-rbac-proxy/0.log"
Jul 28 13:29:10.924554 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:10.924530    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j9sxv_9d4f01c5-13ee-42a1-9969-28b68e374307/exporter/0.log"
Jul 28 13:29:10.945692 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:10.945666    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j9sxv_9d4f01c5-13ee-42a1-9969-28b68e374307/extractor/0.log"
Jul 28 13:29:10.952412 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:10.952394    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5hhgv/perf-node-gather-daemonset-bwvzw"
Jul 28 13:29:13.506952 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:13.506918    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-flt4v_033608ea-141b-49ce-a3c7-b9b5e2c43924/manager/0.log"
Jul 28 13:29:13.911452 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:13.911368    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rx4f2_288e06ff-932a-4134-80a1-3fe602c1e6dd/manager/0.log"
Jul 28 13:29:14.168109 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:14.168005    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65dd8bc54b-vnqz4_a55f2b36-a833-46cf-8c4a-de402175e5a1/manager/0.log"
Jul 28 13:29:14.186942 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:14.186907    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-f26sg_0f12ed3b-d534-49ed-996b-d079bfaa4d0d/server/0.log"
Jul 28 13:29:14.307153 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:14.307122    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cbdrp_9c6e1025-9697-41d3-a393-b3d8e84b5098/manager/0.log"
Jul 28 13:29:14.330456 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:14.330430    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7974886bdb-2rkn7_7b7043db-c8ad-4133-9d65-69ea83a34fe5/manager/0.log"
Jul 28 13:29:14.350421 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:14.350396    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7974886bdb-2rkn7_7b7043db-c8ad-4133-9d65-69ea83a34fe5/copy-manifests/0.log"
Jul 28 13:29:15.375438 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:15.375350    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29_bdb1d50f-0db7-4127-aa3a-65dad7bb40af/istio-proxy/0.log"
Jul 28 13:29:15.648765 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:15.648686    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b958f484-9pbr5_6187130a-311c-4bf7-88e1-18caab96d7e7/kube-auth-proxy/0.log"
Jul 28 13:29:15.688026 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:15.687997    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j4qxs_db993d21-5f17-4559-94b3-075a8a6acac2/istio-proxy/0.log"
Jul 28 13:29:15.741961 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:15.741931    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j4qxs_db993d21-5f17-4559-94b3-075a8a6acac2/istio-proxy/1.log"
Jul 28 13:29:15.770340 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:15.770312    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-r4f55_a7bfe86a-b217-4873-88b7-052fd8b09ad2/payload-pre-processing/0.log"
Jul 28 13:29:15.950772 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:15.950663    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-v2bp4_1f938f96-5882-4431-9e61-ec95948a57b5/openshift-lws-operator/0.log"
Jul 28 13:29:17.208899 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:17.208861    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rls29_bdb1d50f-0db7-4127-aa3a-65dad7bb40af/istio-proxy/0.log"
Jul 28 13:29:17.492426 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:17.492356    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b958f484-9pbr5_6187130a-311c-4bf7-88e1-18caab96d7e7/kube-auth-proxy/0.log"
Jul 28 13:29:17.531901 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:17.531868    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j4qxs_db993d21-5f17-4559-94b3-075a8a6acac2/istio-proxy/0.log"
Jul 28 13:29:17.586001 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:17.585968    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j4qxs_db993d21-5f17-4559-94b3-075a8a6acac2/istio-proxy/1.log"
Jul 28 13:29:17.621741 ip-10-0-132-90 kubenswrapper[2549]: I0728 13:29:17.621718    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-r4f55_a7bfe86a-b217-4873-88b7-052fd8b09ad2/payload-pre-processing/0.log"