Jul 30 20:25:21.424981 ip-10-0-134-148 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 20:25:21.832013 ip-10-0-134-148 kubenswrapper[2547]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 20:25:21.832013 ip-10-0-134-148 kubenswrapper[2547]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 20:25:21.832013 ip-10-0-134-148 kubenswrapper[2547]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 20:25:21.832013 ip-10-0-134-148 kubenswrapper[2547]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 20:25:21.832013 ip-10-0-134-148 kubenswrapper[2547]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 20:25:21.832567 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.832067    2547 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 20:25:21.838989 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.838969    2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 20:25:21.838989 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.838983    2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 20:25:21.838989 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.838987    2547 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 20:25:21.838989 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.838990    2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 20:25:21.838989 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.838993    2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.838996    2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.838999    2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839001    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839004    2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839007    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839009    2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839012    2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839014    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839017    2547 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839020    2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839022    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839025    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839027    2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839029    2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839032    2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839038    2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839041    2547 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839043    2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839045    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 20:25:21.839137 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839048    2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839050    2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839053    2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839056    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839058    2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839061    2547 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839064    2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839066    2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839071    2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839075    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839077    2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839080    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839083    2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839085    2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839088    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839092    2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839096    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839099    2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839102    2547 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 20:25:21.839671 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839105    2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839107    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839110    2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839112    2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839115    2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839117    2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839121    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839123    2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839126    2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839128    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839132    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839134    2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839137    2547 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839140    2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839143    2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839146    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839148    2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839151    2547 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839155    2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839157    2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 20:25:21.840334 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839160    2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839162    2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839165    2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839167    2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839171    2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839173    2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839176    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839179    2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839182    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839184    2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839187    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839189    2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839192    2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839194    2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839196    2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839199    2547 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839201    2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839204    2547 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839206    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 20:25:21.840889 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839208    2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839211    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839213    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839216    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839219    2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839221    2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839225    2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839229    2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839231    2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839234    2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839236    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839239    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839242    2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839244    2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839247    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839249    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839252    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839931    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839937    2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 20:25:21.841387 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839940    2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839956    2547 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839960    2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839962    2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839965    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839969    2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839973    2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839976    2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839979    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839982    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839984    2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839987    2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839989    2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839992    2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839995    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.839997    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840000    2547 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840003    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840006    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 20:25:21.841935 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840009    2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840011    2547 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840014    2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840017    2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840020    2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840022    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840025    2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840027    2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840030    2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840032    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840034    2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840037    2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840039    2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840041    2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840045    2547 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840047    2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840050    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840052    2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840055    2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840057    2547 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 20:25:21.842438 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840060    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840062    2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840065    2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840067    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840069    2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840072    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840075    2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840077    2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840080    2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840082    2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840085    2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840087    2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840091    2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840095    2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840098    2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840100    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840103    2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840105    2547 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840108    2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840110    2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 20:25:21.843023 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840112    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840115    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840118    2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840120    2547 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840122    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840125    2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840128    2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840130    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840133    2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840135    2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840137    2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840140    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840142    2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840144    2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840148    2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840150    2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840153    2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840155    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840159    2547 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840161    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 20:25:21.843584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840164    2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840166    2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840168    2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840171    2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840174    2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840181    2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840184    2547 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840186    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840189    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840192    2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840194    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840197    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840199    2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840202    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840204    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840207    2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840209    2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840212    2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840549    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 20:25:21.844391 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840554    2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840556    2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840559    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840562    2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840565    2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840567    2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840570    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840572    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840575    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840578    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840580    2547 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840583    2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840586    2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840588    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840591    2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840593    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840596    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840598    2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840601    2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840604    2547 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 20:25:21.844924 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840606    2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840609    2547 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840612    2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840615    2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840618    2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840620    2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840622    2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840625    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840627    2547 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840629    2547 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840632    2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840634    2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840637    2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840639    2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840641    2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840644    2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840646    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840649    2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840651    2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840654    2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 20:25:21.845613 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840656    2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840658    2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840661    2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840663    2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840665    2547 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840668    2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840670    2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840673    2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840675    2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840677    2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840679    2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840682    2547 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840685    2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840687    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840690    2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840693    2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840712    2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840720    2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840726    2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840731    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 20:25:21.846239 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840734    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840737    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840740    2547 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840744    2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840747    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840750    2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840752    2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840755    2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840757    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840760    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840762    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840766    2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840768    2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840771    2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840773    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840776    2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840778    2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840781    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840783    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 20:25:21.846850 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840786    2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840788    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840790    2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840793    2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840795    2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840799    2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840801    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840804    2547 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840807    2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840809    2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840812    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840814    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840816    2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840819    2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840821    2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840824    2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840826    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840829    2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.840831    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840902    2547 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840918    2547 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 20:25:21.847364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840924    2547 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840928    2547 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840932    2547 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840935    2547 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840939    2547 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840942    2547 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840945    2547 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840948    2547 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840951    2547 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840954    2547 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840957    2547 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840960    2547 flags.go:64] FLAG: --cgroup-root=""
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840963    2547 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840966    2547 flags.go:64] FLAG: --client-ca-file=""
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840969    2547 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840971    2547 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840976    2547 flags.go:64] FLAG: --cluster-domain=""
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840979    2547 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840982    2547 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840987    2547 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840990    2547 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840993    2547 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840996    2547 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 20:25:21.847951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.840999    2547 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841002    2547 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841005    2547 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841008    2547 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841010    2547 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841013    2547 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841016    2547 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841020    2547 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841022    2547 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841025    2547 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841028    2547 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841031    2547 flags.go:64] FLAG: --enable-server="true"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841033    2547 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841037    2547 flags.go:64] FLAG: --event-burst="100"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841040    2547 flags.go:64] FLAG: --event-qps="50"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841044    2547 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841047    2547 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841049    2547 flags.go:64] FLAG: --eviction-hard=""
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841053    2547 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841056    2547 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841058    2547 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841061    2547 flags.go:64] FLAG: --eviction-soft=""
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841064    2547 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841067    2547 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841070    2547 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 20:25:21.848548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841072    2547 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841075    2547 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841081    2547 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841084    2547 flags.go:64] FLAG: --feature-gates=""
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841088    2547 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841091    2547 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841093    2547 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841096    2547 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841099    2547 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841102    2547 flags.go:64] FLAG: --help="false"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841105    2547 flags.go:64] FLAG: --hostname-override="ip-10-0-134-148.ec2.internal"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841108    2547 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841111    2547 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841114    2547 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841117    2547 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841120    2547 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841123    2547 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841125    2547 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841128    2547 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841131    2547 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841133    2547 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841137    2547 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841141    2547 flags.go:64] FLAG: --kube-reserved=""
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841144    2547 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 20:25:21.849286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841146    2547 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841149    2547 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841152    2547 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841155    2547 flags.go:64] FLAG: --lock-file=""
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841157    2547 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841160    2547 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841163    2547 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841168    2547 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841171    2547 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841173    2547 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841176    2547 flags.go:64] FLAG: --logging-format="text"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841181    2547 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841184    2547 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841187    2547 flags.go:64] FLAG: --manifest-url=""
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841189    2547 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841194    2547 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841197    2547 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841200    2547 flags.go:64] FLAG: --max-pods="110"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841203    2547 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841206    2547 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841208    2547 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841211    2547 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841214    2547 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841217    2547 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841219    2547 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 20:25:21.850021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841225    2547 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841228    2547 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841231    2547 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841234    2547 flags.go:64] FLAG: --pod-cidr=""
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841236    2547 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841241    2547 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841245    2547 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841248    2547 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841251    2547 flags.go:64] FLAG: --port="10250"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841254    2547 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841257    2547 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04991d82d38eac8d0"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841260    2547 flags.go:64] FLAG: --qos-reserved=""
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841262    2547 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841265    2547 flags.go:64] FLAG: --register-node="true"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841268    2547 flags.go:64] FLAG: --register-with-taints=""
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841271    2547 flags.go:64] FLAG: --registry-burst="10"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841274    2547 flags.go:64] FLAG: --registry-qps="5"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841276    2547 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841280    2547 flags.go:64] FLAG: --reserved-memory=""
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841284    2547 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841287    2547 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841289    2547 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841292    2547 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841295    2547 flags.go:64] FLAG: --runonce="false"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841297    2547 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 20:25:21.850669 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841300    2547 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841303    2547 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841306    2547 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841308    2547 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841311    2547 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841314    2547 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841317    2547 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841319    2547 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841322    2547 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841324    2547 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841327    2547 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841330    2547 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841332    2547 flags.go:64] FLAG: --system-cgroups=""
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841336    2547 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841341    2547 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841344    2547 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841347    2547 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841351    2547 flags.go:64] FLAG: --tls-min-version=""
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841354    2547 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841356    2547 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841359    2547 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841361    2547 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841364    2547 flags.go:64] FLAG: --v="2"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841367    2547 flags.go:64] FLAG: --version="false"
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841371    2547 flags.go:64] FLAG: --vmodule=""
Jul 30 20:25:21.851394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841375    2547 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.841378    2547 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841508    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841512    2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841515    2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841518    2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841521    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841524    2547 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841527    2547 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841529    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841532    2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841535    2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841537    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841540    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841542    2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841545    2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841547    2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841550    2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841552    2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841555    2547 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 20:25:21.852068 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841558    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841561    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841563    2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841566    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841568    2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841571    2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841573    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841576    2547 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841578    2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841581    2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841583    2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841586    2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841589    2547 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841592    2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841594    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841597    2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841600    2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841602    2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841605    2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841607    2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 20:25:21.852602 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841609    2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841612    2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841614    2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841616    2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841619    2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841621    2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841624    2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841626    2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841629    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841632    2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841637    2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841639    2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841643    2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841646    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841648    2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841651    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841653    2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841656    2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841659    2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 20:25:21.853184 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841661    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841664    2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841666    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841668    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841671    2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841674    2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841677    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841679    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841682    2547 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841685    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841687    2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841689    2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841692    2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841694    2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841711    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841714    2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841717    2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841719    2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841722    2547 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841724    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 20:25:21.853713 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841727    2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841730    2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841732    2547 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841734    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841737    2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841742    2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841745    2547 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841748    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841751    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841754    2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841758    2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841760    2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841763    2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841766    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841768    2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841771    2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841773    2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841777    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841779    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 20:25:21.854238 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841782    2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841784    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.841786    2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.842393    2547 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.851090    2547 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.851105    2547 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.851893    2547 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.851901    2547 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851963    2547 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851968    2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851972    2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851975    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851978    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851980    2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851983    2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 20:25:21.854813 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851986    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851988    2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851991    2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851994    2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.851996    2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852000    2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852003    2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852006    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852009    2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852012    2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852015    2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852017    2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852020    2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852023    2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852025    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852029    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852031    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852034    2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852036    2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 20:25:21.855217 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852039    2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852041    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852044    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852048    2547 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852051    2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852054    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852057    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852060    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852062    2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852065    2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852067    2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852070    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852072    2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852075    2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852078    2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852080    2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852084    2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852088    2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852090    2547 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 20:25:21.855735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852093    2547 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852095    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852098    2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852101    2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852103    2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852106    2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852108    2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852111    2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852113    2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852115    2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852117    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852120    2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852122    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852125    2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852127    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852130    2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852133    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852135    2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852138    2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852141    2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 20:25:21.856259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852143    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852146    2547 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852148    2547 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852150    2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852153    2547 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852155    2547 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852158    2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852160    2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852163    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852165    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852168    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852170    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852172    2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852175    2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852178    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852181    2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852184    2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852186    2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852188    2547 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852191    2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 20:25:21.856839 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852193    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852196    2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852198    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852200    2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852203    2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852206    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852208    2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852212    2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852215    2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852218    2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852221    2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852224    2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852227    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852229    2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.852250    2547 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 20:25:21.857375 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852357    2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852362    2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852365    2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852368    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852371    2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852373    2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852376    2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852379    2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852381    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852384    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852386    2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852389    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852392    2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852395    2547 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852398    2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852400    2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852402    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852406    2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852410    2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 20:25:21.857797 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852413    2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852415    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852418    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852420    2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852423    2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852425    2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852428    2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852431    2547 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852433    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852436    2547 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852439    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852441    2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852443    2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852446    2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852448    2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852451    2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852453    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852455    2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852458    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852460    2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 20:25:21.858293 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852463    2547 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852465    2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852468    2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852470    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852472    2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852475    2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852478    2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852481    2547 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852483    2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852486    2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852488    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852491    2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852493    2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852495    2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852498    2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852500    2547 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852503    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852505    2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852508    2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852510    2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 20:25:21.858903 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852513    2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852515    2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852518    2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852520    2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852522    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852525    2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852527    2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852530    2547 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852532    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852534    2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852537    2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852539    2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852541    2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852544    2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852546    2547 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852548    2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852551    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852553    2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852556    2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852558    2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 20:25:21.859434 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852561    2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852563    2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852565    2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852568    2547 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852570    2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852573    2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852575    2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852578    2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852580    2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852582    2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852585    2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852587    2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852589    2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852593    2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852596    2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852599    2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852602    2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852605    2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852607    2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 20:25:21.860026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:21.852611    2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 20:25:21.860515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.852616    2547 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 20:25:21.860515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.852740    2547 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 20:25:21.860515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.854692    2547 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 20:25:21.860515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.856257    2547 server.go:1019] "Starting client certificate rotation"
Jul 30 20:25:21.860515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.856359    2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 20:25:21.860515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.856394    2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 20:25:21.860515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.858652    2547 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 20:25:21.860515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.859724    2547 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 20:25:21.871987 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.871970    2547 log.go:25] "Validated CRI v1 runtime API"
Jul 30 20:25:21.874176 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.874162    2547 log.go:25] "Validated CRI v1 image API"
Jul 30 20:25:21.874945 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.874929    2547 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 20:25:21.877744 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.877715    2547 fs.go:135] Filesystem UUIDs: map[1e9d95df-9e08-431f-a2fc-fefa6490d3ea:/dev/nvme0n1p3 5ff6f526-be58-4c96-b901-13f35b18f9ad:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 30 20:25:21.877820 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.877743    2547 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 20:25:21.884200 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.884080    2547 manager.go:216] Machine: {Timestamp:2026-07-30 20:25:21.882751708 +0000 UTC m=+0.343710511 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100069 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29b6869b4970f859c0f59dffc1e711 SystemUUID:ec29b686-9b49-70f8-59c0-f59dffc1e711 BootID:0d0babc7-82d8-4f58-a651-80a8339afb6c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:ea:80:b6:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:ea:80:b6:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:ae:d4:e7:af:ad Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 20:25:21.884200 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.884187    2547 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 20:25:21.884317 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.884263    2547 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 20:25:21.886459 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.886442    2547 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 20:25:21.887421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.887401    2547 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 20:25:21.887542 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.887424    2547 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-148.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 20:25:21.888149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.888139    2547 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 20:25:21.888189 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.888152    2547 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 20:25:21.888189 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.888159    2547 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 20:25:21.889017 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.889007    2547 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 20:25:21.889908 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.889899    2547 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 20:25:21.893552 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.893541    2547 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 20:25:21.893668 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.893658    2547 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 20:25:21.893711 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.893687    2547 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 20:25:21.893752 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.893725    2547 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 20:25:21.893752 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.893735    2547 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 20:25:21.893752 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.893744    2547 kubelet.go:398] "Adding apiserver pod source"
Jul 30 20:25:21.893872 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.893754    2547 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 20:25:21.894056 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.894045    2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 20:25:21.894087 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.894065    2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 20:25:21.896555 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.896540    2547 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 20:25:21.897755 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.897742    2547 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 20:25:21.897803 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.897772    2547 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 20:25:21.899084 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899071    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899087    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899093    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899098    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899106    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899112    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899120    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899126    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899132    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899137    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899150    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 20:25:21.899152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899154    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 20:25:21.899871 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899858    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 20:25:21.899871 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.899868    2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 20:25:21.902759 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.902745    2547 server.go:1268] "Started kubelet"
Jul 30 20:25:21.902875 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.902821    2547 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 20:25:21.902875 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.902848    2547 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 20:25:21.903268 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.903251    2547 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 20:25:21.903528 ip-10-0-134-148 systemd[1]: Started Kubernetes Kubelet.
Jul 30 20:25:21.904199 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.904086    2547 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 20:25:21.904199 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.904129    2547 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-148.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 20:25:21.904199 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.904149    2547 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 20:25:21.904459 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.904419    2547 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 20:25:21.905757 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.905737    2547 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 20:25:21.906986 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.906969    2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-65h4f"
Jul 30 20:25:21.908043 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.908023    2547 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 20:25:21.909086 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.908001    2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-148.ec2.internal.18c72b0f47ad12ff  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-148.ec2.internal,UID:ip-10-0-134-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-148.ec2.internal,},FirstTimestamp:2026-07-30 20:25:21.902719743 +0000 UTC m=+0.363678547,LastTimestamp:2026-07-30 20:25:21.902719743 +0000 UTC m=+0.363678547,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-148.ec2.internal,}"
Jul 30 20:25:21.909634 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.909622    2547 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 20:25:21.909694 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.909642    2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 20:25:21.909813 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.909757    2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:21.910196 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.910181    2547 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 20:25:21.910290 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.910195    2547 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 20:25:21.910350 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.910341    2547 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 20:25:21.910585 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.910568    2547 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 20:25:21.910585 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.910582    2547 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 20:25:21.910847 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.910834    2547 factory.go:55] Registering systemd factory
Jul 30 20:25:21.910847 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.910847    2547 factory.go:223] Registration of the systemd container factory successfully
Jul 30 20:25:21.911082 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.911069    2547 factory.go:153] Registering CRI-O factory
Jul 30 20:25:21.911139 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.911085    2547 factory.go:223] Registration of the crio container factory successfully
Jul 30 20:25:21.911139 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.911137    2547 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 20:25:21.911224 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.911164    2547 factory.go:103] Registering Raw factory
Jul 30 20:25:21.911224 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.911177    2547 manager.go:1196] Started watching for new ooms in manager
Jul 30 20:25:21.911855 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.911840    2547 manager.go:349] Starting recovery of all containers
Jul 30 20:25:21.915963 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.915928    2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-65h4f"
Jul 30 20:25:21.916171 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.916150    2547 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 20:25:21.916253 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.916205    2547 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 20:25:21.925282 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.925146    2547 manager.go:354] Recovery completed
Jul 30 20:25:21.929451 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.929440    2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 20:25:21.931257 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.931239    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 20:25:21.931340 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.931266    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 20:25:21.931340 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.931276    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientPID"
Jul 30 20:25:21.931909 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.931893    2547 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 20:25:21.931909 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.931906    2547 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 20:25:21.932057 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.931920    2547 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 20:25:21.932057 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.931920    2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 20:25:21.932612 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.932595    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 20:25:21.932680 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.932620    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 20:25:21.932680 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.932630    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientPID"
Jul 30 20:25:21.933352 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.933340    2547 policy_none.go:49] "None policy: Start"
Jul 30 20:25:21.933408 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.933356    2547 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 20:25:21.933408 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.933366    2547 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 20:25:21.933857 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.933783    2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-148.ec2.internal.18c72b0f49607aa0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-148.ec2.internal,UID:ip-10-0-134-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-148.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-148.ec2.internal,},FirstTimestamp:2026-07-30 20:25:21.931254432 +0000 UTC m=+0.392213235,LastTimestamp:2026-07-30 20:25:21.931254432 +0000 UTC m=+0.392213235,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-148.ec2.internal,}"
Jul 30 20:25:21.934373 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.934361    2547 policy_none.go:47] "Start"
Jul 30 20:25:21.965272 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.965251    2547 manager.go:337] "Starting Device Plugin manager"
Jul 30 20:25:21.965359 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.965284    2547 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 20:25:21.965359 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.965295    2547 server.go:87] "Starting device plugin registration server"
Jul 30 20:25:21.965525 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.965513    2547 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 20:25:21.965580 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.965540    2547 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 20:25:21.965651 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.965632    2547 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 20:25:21.965734 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.965723    2547 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 20:25:21.965734 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:21.965732    2547 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 20:25:21.966274 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.966259    2547 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 20:25:21.966336 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:21.966303    2547 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.064657 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.064623    2547 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 20:25:22.065841 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.065826    2547 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 20:25:22.065915 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.065856    2547 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 20:25:22.065915 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.065874    2547 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 20:25:22.065915 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.065908    2547 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 20:25:22.065915 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.065832    2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 20:25:22.066746 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.066726    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 20:25:22.066851 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.066760    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 20:25:22.066851 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.066774    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientPID"
Jul 30 20:25:22.066851 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.066794    2547 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.068280 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.068266    2547 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 20:25:22.075112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.075092    2547 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.075112 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.075112    2547 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-148.ec2.internal\": node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.091546 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.091509    2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.166208 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.166186    2547 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal"]
Jul 30 20:25:22.166278 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.166240    2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 20:25:22.167534 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.167511    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 20:25:22.167631 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.167543    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 20:25:22.167631 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.167558    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientPID"
Jul 30 20:25:22.168711 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.168686    2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 20:25:22.168799 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.168786    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.168844 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.168815    2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 20:25:22.169400 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.169383    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 20:25:22.169469 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.169415    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 20:25:22.169469 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.169425    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientPID"
Jul 30 20:25:22.169469 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.169383    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 20:25:22.169575 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.169493    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 20:25:22.169575 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.169505    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientPID"
Jul 30 20:25:22.170778 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.170761    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.170856 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.170788    2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 20:25:22.171457 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.171443    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 20:25:22.171521 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.171471    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 20:25:22.171521 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.171483    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeHasSufficientPID"
Jul 30 20:25:22.192062 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.192042    2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.195332 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.195314    2547 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-148.ec2.internal\" not found" node="ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.199854 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.199838    2547 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-148.ec2.internal\" not found" node="ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.211452 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.211427    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05a45d6bd57d0022f8b25ea0fa9c160f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal\" (UID: \"05a45d6bd57d0022f8b25ea0fa9c160f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.211543 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.211458    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/858f9e0ac50469cf471404be973e16f5-config\") pod \"kube-apiserver-proxy-ip-10-0-134-148.ec2.internal\" (UID: \"858f9e0ac50469cf471404be973e16f5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.211543 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.211474    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05a45d6bd57d0022f8b25ea0fa9c160f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal\" (UID: \"05a45d6bd57d0022f8b25ea0fa9c160f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.293164 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.293144    2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.312509 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.312490    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05a45d6bd57d0022f8b25ea0fa9c160f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal\" (UID: \"05a45d6bd57d0022f8b25ea0fa9c160f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.312580 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.312520    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05a45d6bd57d0022f8b25ea0fa9c160f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal\" (UID: \"05a45d6bd57d0022f8b25ea0fa9c160f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.312580 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.312536    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/858f9e0ac50469cf471404be973e16f5-config\") pod \"kube-apiserver-proxy-ip-10-0-134-148.ec2.internal\" (UID: \"858f9e0ac50469cf471404be973e16f5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.312580 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.312576    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05a45d6bd57d0022f8b25ea0fa9c160f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal\" (UID: \"05a45d6bd57d0022f8b25ea0fa9c160f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.312932 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.312893    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/05a45d6bd57d0022f8b25ea0fa9c160f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal\" (UID: \"05a45d6bd57d0022f8b25ea0fa9c160f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.313185 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.313149    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/858f9e0ac50469cf471404be973e16f5-config\") pod \"kube-apiserver-proxy-ip-10-0-134-148.ec2.internal\" (UID: \"858f9e0ac50469cf471404be973e16f5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.393773 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.393717    2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.494351 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.494326    2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.498493 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.498478    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.503036 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.502837    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.594617 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.594591    2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.695085 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.695031    2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.795603 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.795579    2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-148.ec2.internal\" not found"
Jul 30 20:25:22.852601 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.852573    2547 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 20:25:22.855712 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.855675    2547 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 20:25:22.855852 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.855832    2547 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 20:25:22.855920 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.855893    2547 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 20:25:22.894004 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.893979    2547 apiserver.go:52] "Watching apiserver"
Jul 30 20:25:22.910618 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.910594    2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 20:25:22.910728 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.910622    2547 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.912790 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.912774    2547 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 20:25:22.913889 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.913869    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-9ttmg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss","openshift-image-registry/node-ca-8kxwd","openshift-multus/multus-additional-cni-plugins-2s59d","openshift-multus/network-metrics-daemon-l6rqg","openshift-network-diagnostics/network-check-target-wfrtr","openshift-network-operator/iptables-alerter-sv6zd","kube-system/konnectivity-agent-qbvzb","openshift-cluster-node-tuning-operator/tuned-dgwck","openshift-dns/node-resolver-8x4qs","openshift-multus/multus-jvrq2","openshift-ovn-kubernetes/ovnkube-node-clxfr"]
Jul 30 20:25:22.915275 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.915260    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:22.915346 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.915323    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:22.916554 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.916534    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:22.917763 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.917745    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:22.918875 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.918853    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:22.918975 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.918922    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 20:25:22.919248 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.919225    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bhccz\""
Jul 30 20:25:22.919355 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.919274    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 20:25:22.919413 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.919382    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 20:25:22.920261 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.920203    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:22.920261 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.920210    2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 20:20:22 +0000 UTC" deadline="2028-01-04 00:01:23.195303404 +0000 UTC"
Jul 30 20:25:22.920261 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.920233    2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12531h36m0.275073268s"
Jul 30 20:25:22.920460 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.920267    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:22.920460 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.920283    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 20:25:22.920548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.920456    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 20:25:22.920548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.920456    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 20:25:22.920629 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.920558    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qpn86\""
Jul 30 20:25:22.921627 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.921597    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 20:25:22.921738 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.921667    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 20:25:22.921802 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.921748    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q4kff\""
Jul 30 20:25:22.921889 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.921871    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:22.921947 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.921898    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 20:25:22.921947 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.921915    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 20:25:22.922061 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:22.921954    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:22.922061 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.922052    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 20:25:22.922977 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.922958    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:22.924167 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.924151    2547 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 20:25:22.924230 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.924214    2547 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 20:25:22.924302 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.924229    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:22.924345 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.924304    2547 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal"
Jul 30 20:25:22.925343 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.925327    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 20:25:22.925464 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.925451    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:22.925618 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.925601    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7pkgc\""
Jul 30 20:25:22.925687 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.925662    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 20:25:22.926562 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.926543    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 20:25:22.927029 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.927011    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:22.927227 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.927209    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5qfsx\""
Jul 30 20:25:22.927331 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.927301    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 20:25:22.928248 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.928229    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 20:25:22.928538 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.928521    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nqhdg\""
Jul 30 20:25:22.928645 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.928523    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 20:25:22.929836 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.929597    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 20:25:22.930164 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.930147    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jglz7\""
Jul 30 20:25:22.930926 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.930907    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:22.932323 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.932298    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal"]
Jul 30 20:25:22.932397 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.932383    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:22.933518 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.933494    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 20:25:22.933518 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.933514    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 20:25:22.933664 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.933573    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wnzl9\""
Jul 30 20:25:22.934910 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.934888    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 20:25:22.934999 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.934933    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 20:25:22.934999 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.934900    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 20:25:22.935365 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.935350    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x8kxw\""
Jul 30 20:25:22.935431 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.935392    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 20:25:22.935484 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.935448    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 20:25:22.935484 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.935452    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 20:25:22.936280 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.936260    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 20:25:22.936439 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.936425    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal"]
Jul 30 20:25:22.936500 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.936475    2547 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 20:25:22.946029 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.946013    2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5bnts"
Jul 30 20:25:22.954168 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.954150    2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5bnts"
Jul 30 20:25:22.976820 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:22.976784    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod858f9e0ac50469cf471404be973e16f5.slice/crio-b4e33c83ee234acdcef8392d70a3e1478fcc951781b1bbcb48f53871c9735d57 WatchSource:0}: Error finding container b4e33c83ee234acdcef8392d70a3e1478fcc951781b1bbcb48f53871c9735d57: Status 404 returned error can't find the container with id b4e33c83ee234acdcef8392d70a3e1478fcc951781b1bbcb48f53871c9735d57
Jul 30 20:25:22.977126 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:22.977107    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a45d6bd57d0022f8b25ea0fa9c160f.slice/crio-2fb9fdde9fe1ed0bb4579efab819ac0784576eba025e7de93373a14619435bcf WatchSource:0}: Error finding container 2fb9fdde9fe1ed0bb4579efab819ac0784576eba025e7de93373a14619435bcf: Status 404 returned error can't find the container with id 2fb9fdde9fe1ed0bb4579efab819ac0784576eba025e7de93373a14619435bcf
Jul 30 20:25:22.981397 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:22.981383    2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:25:23.013290 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.012778    2547 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 20:25:23.015546 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015529    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-cni-bin\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.015622 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015560    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-etc-selinux\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.015622 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015586    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17492ecc-9882-4680-9df3-5ef550df1e0d-host\") pod \"node-ca-8kxwd\" (UID: \"17492ecc-9882-4680-9df3-5ef550df1e0d\") " pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.015622 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015609    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ee03f66-de6a-40c8-a2c0-534ac1754eb2-konnectivity-ca\") pod \"konnectivity-agent-qbvzb\" (UID: \"9ee03f66-de6a-40c8-a2c0-534ac1754eb2\") " pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:23.015798 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015632    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-var-lib-cni-bin\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.015798 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015655    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-var-lib-openvswitch\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.015798 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015679    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-device-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.015798 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015721    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48cbf098-a8a1-4689-9a3c-ea8784346402-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.015798 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015746    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-var-lib-kubelet\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.015798 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015767    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-log-socket\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.015798 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015790    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9bdc8886-b53f-4145-ac66-ac4de098e6e4-env-overrides\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015812    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj7dv\" (UniqueName: \"kubernetes.io/projected/9bdc8886-b53f-4145-ac66-ac4de098e6e4-kube-api-access-sj7dv\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015844    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5b502fc-da85-472b-947f-0a6e7ca5350b-host-slash\") pod \"iptables-alerter-sv6zd\" (UID: \"b5b502fc-da85-472b-947f-0a6e7ca5350b\") " pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015866    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-registration-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015882    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48cbf098-a8a1-4689-9a3c-ea8784346402-cni-binary-copy\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015896    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-sysctl-d\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015927    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/029812bb-2174-49d1-9bc2-9d576b3c2c80-hosts-file\") pod \"node-resolver-8x4qs\" (UID: \"029812bb-2174-49d1-9bc2-9d576b3c2c80\") " pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015961    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-slash\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.015983    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-systemd\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016015    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-os-release\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016037    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-daemon-config\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016059    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52l2v\" (UniqueName: \"kubernetes.io/projected/6db055c7-8e05-44de-8bc5-716d2b6957f2-kube-api-access-52l2v\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016082    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-node-log\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.016104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016104    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-kubelet-config\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016130    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016164    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-tmp\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016186    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029812bb-2174-49d1-9bc2-9d576b3c2c80-tmp-dir\") pod \"node-resolver-8x4qs\" (UID: \"029812bb-2174-49d1-9bc2-9d576b3c2c80\") " pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016208    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-system-cni-dir\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016231    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-socket-dir-parent\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016279    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-run-openvswitch\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016303    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016327    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016351    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxwk\" (UniqueName: \"kubernetes.io/projected/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-kube-api-access-twxwk\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016384    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-sysconfig\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016407    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-sysctl-conf\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016434    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-conf-dir\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016458    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9bdc8886-b53f-4145-ac66-ac4de098e6e4-ovnkube-script-lib\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016497    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-sys-fs\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016522    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-os-release\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.016747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016545    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-run-netns\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016572    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b5b502fc-da85-472b-947f-0a6e7ca5350b-iptables-alerter-script\") pod \"iptables-alerter-sv6zd\" (UID: \"b5b502fc-da85-472b-947f-0a6e7ca5350b\") " pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016595    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17492ecc-9882-4680-9df3-5ef550df1e0d-serviceca\") pod \"node-ca-8kxwd\" (UID: \"17492ecc-9882-4680-9df3-5ef550df1e0d\") " pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016617    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-cnibin\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016640    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-hostroot\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016665    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-system-cni-dir\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016689    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016714    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhcbg\" (UniqueName: \"kubernetes.io/projected/029812bb-2174-49d1-9bc2-9d576b3c2c80-kube-api-access-hhcbg\") pod \"node-resolver-8x4qs\" (UID: \"029812bb-2174-49d1-9bc2-9d576b3c2c80\") " pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016758    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-run-k8s-cni-cncf-io\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016788    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-systemd-units\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016812    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-socket-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016834    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-sys\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016858    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-host\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016881    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016916    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016949    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-dbus\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:23.017697 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016970    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-run\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.016988    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-lib-modules\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017004    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-var-lib-kubelet\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017019    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-cni-dir\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017033    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-etc-openvswitch\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017062    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48cbf098-a8a1-4689-9a3c-ea8784346402-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017092    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017109    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-modprobe-d\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017124    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-kubernetes\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017141    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-kubelet\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017156    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-cni-netd\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017172    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfgs2\" (UniqueName: \"kubernetes.io/projected/b5b502fc-da85-472b-947f-0a6e7ca5350b-kube-api-access-hfgs2\") pod \"iptables-alerter-sv6zd\" (UID: \"b5b502fc-da85-472b-947f-0a6e7ca5350b\") " pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017188    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74htt\" (UniqueName: \"kubernetes.io/projected/17492ecc-9882-4680-9df3-5ef550df1e0d-kube-api-access-74htt\") pod \"node-ca-8kxwd\" (UID: \"17492ecc-9882-4680-9df3-5ef550df1e0d\") " pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017203    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-run-netns\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017222    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k9kt\" (UniqueName: \"kubernetes.io/projected/48cbf098-a8a1-4689-9a3c-ea8784346402-kube-api-access-4k9kt\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017237    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-tuned\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.018202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017251    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl4k9\" (UniqueName: \"kubernetes.io/projected/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-kube-api-access-rl4k9\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017266    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6db055c7-8e05-44de-8bc5-716d2b6957f2-cni-binary-copy\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017280    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-etc-kubernetes\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017293    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-run-systemd\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017323    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-run-ovn\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017352    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9bdc8886-b53f-4145-ac66-ac4de098e6e4-ovnkube-config\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017378    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bdc8886-b53f-4145-ac66-ac4de098e6e4-ovn-node-metrics-cert\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017408    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phjq2\" (UniqueName: \"kubernetes.io/projected/bc16c623-e077-4823-ac5d-d427aa4de728-kube-api-access-phjq2\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017427    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-var-lib-cni-multus\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017443    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-run-multus-certs\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017457    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-cnibin\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.018707 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.017473    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ee03f66-de6a-40c8-a2c0-534ac1754eb2-agent-certs\") pod \"konnectivity-agent-qbvzb\" (UID: \"9ee03f66-de6a-40c8-a2c0-534ac1754eb2\") " pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:23.069353 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.069316    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal" event={"ID":"858f9e0ac50469cf471404be973e16f5","Type":"ContainerStarted","Data":"b4e33c83ee234acdcef8392d70a3e1478fcc951781b1bbcb48f53871c9735d57"}
Jul 30 20:25:23.070474 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.070455    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal" event={"ID":"05a45d6bd57d0022f8b25ea0fa9c160f","Type":"ContainerStarted","Data":"2fb9fdde9fe1ed0bb4579efab819ac0784576eba025e7de93373a14619435bcf"}
Jul 30 20:25:23.078139 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.078124    2547 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 20:25:23.117892 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.117868    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-run-openvswitch\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.117991 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.117897    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.117991 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.117920    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:23.117991 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.117935    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twxwk\" (UniqueName: \"kubernetes.io/projected/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-kube-api-access-twxwk\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:23.118149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.117993    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.118149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118014    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-sysconfig\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.118149 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.118038    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:23.118149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118046    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-sysctl-conf\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.118149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.117996    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-run-openvswitch\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.118149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118066    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-sysconfig\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.118149 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.118098    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret podName:643b0bcf-2222-46a0-b8ba-5d556c74e5a3 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:23.618078028 +0000 UTC m=+2.079036829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret") pod "global-pull-secret-syncer-9ttmg" (UID: "643b0bcf-2222-46a0-b8ba-5d556c74e5a3") : object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:23.118149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118116    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-conf-dir\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.118149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118145    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9bdc8886-b53f-4145-ac66-ac4de098e6e4-ovnkube-script-lib\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118161    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-sysctl-conf\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118171    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-sys-fs\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118178    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-conf-dir\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118229    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-sys-fs\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118278    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-os-release\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118319    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-run-netns\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118325    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-os-release\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118350    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b5b502fc-da85-472b-947f-0a6e7ca5350b-iptables-alerter-script\") pod \"iptables-alerter-sv6zd\" (UID: \"b5b502fc-da85-472b-947f-0a6e7ca5350b\") " pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118378    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17492ecc-9882-4680-9df3-5ef550df1e0d-serviceca\") pod \"node-ca-8kxwd\" (UID: \"17492ecc-9882-4680-9df3-5ef550df1e0d\") " pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118402    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-cnibin\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118427    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-hostroot\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118430    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-run-netns\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118465    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-system-cni-dir\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118479    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-hostroot\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118499    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118510    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-cnibin\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118524    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-system-cni-dir\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.118594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118526    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhcbg\" (UniqueName: \"kubernetes.io/projected/029812bb-2174-49d1-9bc2-9d576b3c2c80-kube-api-access-hhcbg\") pod \"node-resolver-8x4qs\" (UID: \"029812bb-2174-49d1-9bc2-9d576b3c2c80\") " pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118552    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-run-k8s-cni-cncf-io\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118575    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-systemd-units\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118600    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-socket-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118627    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-sys\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118651    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-host\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118675    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-systemd-units\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118677    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118626    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-run-k8s-cni-cncf-io\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118679    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118755    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118782    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-dbus\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118765    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9bdc8886-b53f-4145-ac66-ac4de098e6e4-ovnkube-script-lib\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118804    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-run\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118830    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-lib-modules\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118841    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-sys\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118875    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-var-lib-kubelet\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.119494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118884    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17492ecc-9882-4680-9df3-5ef550df1e0d-serviceca\") pod \"node-ca-8kxwd\" (UID: \"17492ecc-9882-4680-9df3-5ef550df1e0d\") " pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118904    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-cni-dir\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118924    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-var-lib-kubelet\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118930    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-etc-openvswitch\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118939    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-lib-modules\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118954    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48cbf098-a8a1-4689-9a3c-ea8784346402-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118972    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-socket-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118979    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118989    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.118995    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-run\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119011    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b5b502fc-da85-472b-947f-0a6e7ca5350b-iptables-alerter-script\") pod \"iptables-alerter-sv6zd\" (UID: \"b5b502fc-da85-472b-947f-0a6e7ca5350b\") " pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119019    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-modprobe-d\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119015    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-dbus\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119032    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-etc-openvswitch\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119047    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-kubernetes\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.119053    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119075    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-kubelet\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119083    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-cni-dir\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.120383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119088    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-kubernetes\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.119097    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs podName:d5679c74-1e69-4be8-90d8-3bf6aa168ee4 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:23.619081503 +0000 UTC m=+2.080040294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs") pod "network-metrics-daemon-l6rqg" (UID: "d5679c74-1e69-4be8-90d8-3bf6aa168ee4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119050    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-host\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119106    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-modprobe-d\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119111    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-kubelet\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119125    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-cni-netd\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119147    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfgs2\" (UniqueName: \"kubernetes.io/projected/b5b502fc-da85-472b-947f-0a6e7ca5350b-kube-api-access-hfgs2\") pod \"iptables-alerter-sv6zd\" (UID: \"b5b502fc-da85-472b-947f-0a6e7ca5350b\") " pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119164    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74htt\" (UniqueName: \"kubernetes.io/projected/17492ecc-9882-4680-9df3-5ef550df1e0d-kube-api-access-74htt\") pod \"node-ca-8kxwd\" (UID: \"17492ecc-9882-4680-9df3-5ef550df1e0d\") " pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119191    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-run-netns\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119216    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k9kt\" (UniqueName: \"kubernetes.io/projected/48cbf098-a8a1-4689-9a3c-ea8784346402-kube-api-access-4k9kt\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119240    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-run-netns\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119283    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-cni-netd\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119326    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-tuned\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119350    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl4k9\" (UniqueName: \"kubernetes.io/projected/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-kube-api-access-rl4k9\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119366    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6db055c7-8e05-44de-8bc5-716d2b6957f2-cni-binary-copy\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119381    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-etc-kubernetes\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119397    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-run-systemd\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.121270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119424    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-run-ovn\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119430    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48cbf098-a8a1-4689-9a3c-ea8784346402-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119452    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-run-systemd\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119467    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-run-ovn\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119480    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-etc-kubernetes\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119561    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9bdc8886-b53f-4145-ac66-ac4de098e6e4-ovnkube-config\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119592    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bdc8886-b53f-4145-ac66-ac4de098e6e4-ovn-node-metrics-cert\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119620    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phjq2\" (UniqueName: \"kubernetes.io/projected/bc16c623-e077-4823-ac5d-d427aa4de728-kube-api-access-phjq2\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119650    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-var-lib-cni-multus\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119627    2547 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119669    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-run-multus-certs\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119693    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-cnibin\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119739    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ee03f66-de6a-40c8-a2c0-534ac1754eb2-agent-certs\") pod \"konnectivity-agent-qbvzb\" (UID: \"9ee03f66-de6a-40c8-a2c0-534ac1754eb2\") " pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119747    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-var-lib-cni-multus\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119765    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-cni-bin\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119790    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-etc-selinux\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119795    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48cbf098-a8a1-4689-9a3c-ea8784346402-cnibin\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119835    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-run-multus-certs\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119884    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-etc-selinux\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119922    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-cni-bin\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119924    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17492ecc-9882-4680-9df3-5ef550df1e0d-host\") pod \"node-ca-8kxwd\" (UID: \"17492ecc-9882-4680-9df3-5ef550df1e0d\") " pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.119980    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ee03f66-de6a-40c8-a2c0-534ac1754eb2-konnectivity-ca\") pod \"konnectivity-agent-qbvzb\" (UID: \"9ee03f66-de6a-40c8-a2c0-534ac1754eb2\") " pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120005    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-var-lib-cni-bin\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120025    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17492ecc-9882-4680-9df3-5ef550df1e0d-host\") pod \"node-ca-8kxwd\" (UID: \"17492ecc-9882-4680-9df3-5ef550df1e0d\") " pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120032    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-var-lib-openvswitch\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120059    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-device-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120064    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6db055c7-8e05-44de-8bc5-716d2b6957f2-cni-binary-copy\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120086    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48cbf098-a8a1-4689-9a3c-ea8784346402-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120142    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-var-lib-openvswitch\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120161    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-var-lib-cni-bin\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120193    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-device-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120204    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9bdc8886-b53f-4145-ac66-ac4de098e6e4-ovnkube-config\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120228    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-var-lib-kubelet\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120264    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-host-var-lib-kubelet\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120267    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-log-socket\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.122766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120304    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9bdc8886-b53f-4145-ac66-ac4de098e6e4-env-overrides\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120333    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj7dv\" (UniqueName: \"kubernetes.io/projected/9bdc8886-b53f-4145-ac66-ac4de098e6e4-kube-api-access-sj7dv\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120381    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5b502fc-da85-472b-947f-0a6e7ca5350b-host-slash\") pod \"iptables-alerter-sv6zd\" (UID: \"b5b502fc-da85-472b-947f-0a6e7ca5350b\") " pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120794    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9bdc8886-b53f-4145-ac66-ac4de098e6e4-env-overrides\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120850    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5b502fc-da85-472b-947f-0a6e7ca5350b-host-slash\") pod \"iptables-alerter-sv6zd\" (UID: \"b5b502fc-da85-472b-947f-0a6e7ca5350b\") " pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120896    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-log-socket\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.120936    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48cbf098-a8a1-4689-9a3c-ea8784346402-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121042    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-registration-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121253    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-registration-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121302    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48cbf098-a8a1-4689-9a3c-ea8784346402-cni-binary-copy\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121344    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-sysctl-d\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121254    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ee03f66-de6a-40c8-a2c0-534ac1754eb2-konnectivity-ca\") pod \"konnectivity-agent-qbvzb\" (UID: \"9ee03f66-de6a-40c8-a2c0-534ac1754eb2\") " pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121373    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/029812bb-2174-49d1-9bc2-9d576b3c2c80-hosts-file\") pod \"node-resolver-8x4qs\" (UID: \"029812bb-2174-49d1-9bc2-9d576b3c2c80\") " pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121419    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-slash\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121446    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-systemd\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121473    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-os-release\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121486    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-sysctl-d\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.123441 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121497    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-daemon-config\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121529    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52l2v\" (UniqueName: \"kubernetes.io/projected/6db055c7-8e05-44de-8bc5-716d2b6957f2-kube-api-access-52l2v\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121536    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/029812bb-2174-49d1-9bc2-9d576b3c2c80-hosts-file\") pod \"node-resolver-8x4qs\" (UID: \"029812bb-2174-49d1-9bc2-9d576b3c2c80\") " pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121552    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-systemd\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121553    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-node-log\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121596    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-kubelet-config\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121623    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121649    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-tmp\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121674    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029812bb-2174-49d1-9bc2-9d576b3c2c80-tmp-dir\") pod \"node-resolver-8x4qs\" (UID: \"029812bb-2174-49d1-9bc2-9d576b3c2c80\") " pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121700    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-system-cni-dir\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121748    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-socket-dir-parent\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121866    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc16c623-e077-4823-ac5d-d427aa4de728-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121909    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-socket-dir-parent\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121911    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48cbf098-a8a1-4689-9a3c-ea8784346402-cni-binary-copy\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121929    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-host-slash\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121979    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-kubelet-config\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.121598    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9bdc8886-b53f-4145-ac66-ac4de098e6e4-node-log\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.122032    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-system-cni-dir\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.124009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.122085    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6db055c7-8e05-44de-8bc5-716d2b6957f2-os-release\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.124587 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.122207    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029812bb-2174-49d1-9bc2-9d576b3c2c80-tmp-dir\") pod \"node-resolver-8x4qs\" (UID: \"029812bb-2174-49d1-9bc2-9d576b3c2c80\") " pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.124587 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.122733    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6db055c7-8e05-44de-8bc5-716d2b6957f2-multus-daemon-config\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.124587 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.123216    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-etc-tuned\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.124587 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.123326    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bdc8886-b53f-4145-ac66-ac4de098e6e4-ovn-node-metrics-cert\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.124587 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.123470    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ee03f66-de6a-40c8-a2c0-534ac1754eb2-agent-certs\") pod \"konnectivity-agent-qbvzb\" (UID: \"9ee03f66-de6a-40c8-a2c0-534ac1754eb2\") " pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:23.124587 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.124093    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-tmp\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.128729 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.128665    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxwk\" (UniqueName: \"kubernetes.io/projected/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-kube-api-access-twxwk\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:23.129277 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.129256    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 20:25:23.129367 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.129283    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 20:25:23.129367 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.129298    2547 projected.go:196] Error preparing data for projected volume kube-api-access-wpz7j for pod openshift-network-diagnostics/network-check-target-wfrtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:23.129367 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.129358    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j podName:b1a23cb4-9909-4e5b-9e7d-79e34f02bb90 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:23.629342151 +0000 UTC m=+2.090300966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wpz7j" (UniqueName: "kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j") pod "network-check-target-wfrtr" (UID: "b1a23cb4-9909-4e5b-9e7d-79e34f02bb90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:23.130275 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.130245    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhcbg\" (UniqueName: \"kubernetes.io/projected/029812bb-2174-49d1-9bc2-9d576b3c2c80-kube-api-access-hhcbg\") pod \"node-resolver-8x4qs\" (UID: \"029812bb-2174-49d1-9bc2-9d576b3c2c80\") " pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.131138 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.131105    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74htt\" (UniqueName: \"kubernetes.io/projected/17492ecc-9882-4680-9df3-5ef550df1e0d-kube-api-access-74htt\") pod \"node-ca-8kxwd\" (UID: \"17492ecc-9882-4680-9df3-5ef550df1e0d\") " pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.131990 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.131966    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfgs2\" (UniqueName: \"kubernetes.io/projected/b5b502fc-da85-472b-947f-0a6e7ca5350b-kube-api-access-hfgs2\") pod \"iptables-alerter-sv6zd\" (UID: \"b5b502fc-da85-472b-947f-0a6e7ca5350b\") " pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.132057 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.131982    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phjq2\" (UniqueName: \"kubernetes.io/projected/bc16c623-e077-4823-ac5d-d427aa4de728-kube-api-access-phjq2\") pod \"aws-ebs-csi-driver-node-d7nss\" (UID: \"bc16c623-e077-4823-ac5d-d427aa4de728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.132318 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.132303    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj7dv\" (UniqueName: \"kubernetes.io/projected/9bdc8886-b53f-4145-ac66-ac4de098e6e4-kube-api-access-sj7dv\") pod \"ovnkube-node-clxfr\" (UID: \"9bdc8886-b53f-4145-ac66-ac4de098e6e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.132742 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.132722    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52l2v\" (UniqueName: \"kubernetes.io/projected/6db055c7-8e05-44de-8bc5-716d2b6957f2-kube-api-access-52l2v\") pod \"multus-jvrq2\" (UID: \"6db055c7-8e05-44de-8bc5-716d2b6957f2\") " pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.133375 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.133356    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl4k9\" (UniqueName: \"kubernetes.io/projected/39a7b62f-3a6f-4cac-b7ac-7e3d6e735318-kube-api-access-rl4k9\") pod \"tuned-dgwck\" (UID: \"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318\") " pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.133449 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.133381    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k9kt\" (UniqueName: \"kubernetes.io/projected/48cbf098-a8a1-4689-9a3c-ea8784346402-kube-api-access-4k9kt\") pod \"multus-additional-cni-plugins-2s59d\" (UID: \"48cbf098-a8a1-4689-9a3c-ea8784346402\") " pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.243849 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.243787    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss"
Jul 30 20:25:23.253208 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:23.253173    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc16c623_e077_4823_ac5d_d427aa4de728.slice/crio-bc0eec18070daa2c1bc8561f63097307cfe1bd953623934d3bff94fb9ce1fbca WatchSource:0}: Error finding container bc0eec18070daa2c1bc8561f63097307cfe1bd953623934d3bff94fb9ce1fbca: Status 404 returned error can't find the container with id bc0eec18070daa2c1bc8561f63097307cfe1bd953623934d3bff94fb9ce1fbca
Jul 30 20:25:23.259172 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.259157    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8kxwd"
Jul 30 20:25:23.266078 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:23.266060    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17492ecc_9882_4680_9df3_5ef550df1e0d.slice/crio-9d12b1b0a585ba52634892ed3c7becb4509e632688932de5633172a16fdc3132 WatchSource:0}: Error finding container 9d12b1b0a585ba52634892ed3c7becb4509e632688932de5633172a16fdc3132: Status 404 returned error can't find the container with id 9d12b1b0a585ba52634892ed3c7becb4509e632688932de5633172a16fdc3132
Jul 30 20:25:23.275500 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.275485    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2s59d"
Jul 30 20:25:23.283446 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.283197    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:23.284010 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:23.283989    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48cbf098_a8a1_4689_9a3c_ea8784346402.slice/crio-8f110caa1a054901bbe81ab083b4d63cd571f0d0b5be7cb8e83d886653803702 WatchSource:0}: Error finding container 8f110caa1a054901bbe81ab083b4d63cd571f0d0b5be7cb8e83d886653803702: Status 404 returned error can't find the container with id 8f110caa1a054901bbe81ab083b4d63cd571f0d0b5be7cb8e83d886653803702
Jul 30 20:25:23.287993 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.287974    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dgwck"
Jul 30 20:25:23.291026 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:23.291007    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee03f66_de6a_40c8_a2c0_534ac1754eb2.slice/crio-6c304d2f15b7e35b7a0f661eb42f52509b1de68d1999ff811a71e219596b8d90 WatchSource:0}: Error finding container 6c304d2f15b7e35b7a0f661eb42f52509b1de68d1999ff811a71e219596b8d90: Status 404 returned error can't find the container with id 6c304d2f15b7e35b7a0f661eb42f52509b1de68d1999ff811a71e219596b8d90
Jul 30 20:25:23.292202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.292183    2547 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 20:25:23.293012 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.292997    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8x4qs"
Jul 30 20:25:23.295879 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:23.295860    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a7b62f_3a6f_4cac_b7ac_7e3d6e735318.slice/crio-a598b479e2b188fb635baaa97922246d8db3a3c6cac9bb99ee83f6399b4b63d6 WatchSource:0}: Error finding container a598b479e2b188fb635baaa97922246d8db3a3c6cac9bb99ee83f6399b4b63d6: Status 404 returned error can't find the container with id a598b479e2b188fb635baaa97922246d8db3a3c6cac9bb99ee83f6399b4b63d6
Jul 30 20:25:23.299674 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.299662    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jvrq2"
Jul 30 20:25:23.303577 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:23.303555    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029812bb_2174_49d1_9bc2_9d576b3c2c80.slice/crio-46a01575270bfa5ecb3b9a4a563a6fa7818839be8b53f0d053c035ba1ef42834 WatchSource:0}: Error finding container 46a01575270bfa5ecb3b9a4a563a6fa7818839be8b53f0d053c035ba1ef42834: Status 404 returned error can't find the container with id 46a01575270bfa5ecb3b9a4a563a6fa7818839be8b53f0d053c035ba1ef42834
Jul 30 20:25:23.304861 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.304843    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:23.309551 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.309530    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sv6zd"
Jul 30 20:25:23.309688 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:23.309668    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db055c7_8e05_44de_8bc5_716d2b6957f2.slice/crio-b9904a2865ac75d5d9832e5c8ad62846060a097518a7558e797a2bdff7e14b14 WatchSource:0}: Error finding container b9904a2865ac75d5d9832e5c8ad62846060a097518a7558e797a2bdff7e14b14: Status 404 returned error can't find the container with id b9904a2865ac75d5d9832e5c8ad62846060a097518a7558e797a2bdff7e14b14
Jul 30 20:25:23.313925 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:23.313902    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdc8886_b53f_4145_ac66_ac4de098e6e4.slice/crio-3382bbb6e5829428cecc528c12cde2d1a1b003b86ab29325472bbba8a5cd5d38 WatchSource:0}: Error finding container 3382bbb6e5829428cecc528c12cde2d1a1b003b86ab29325472bbba8a5cd5d38: Status 404 returned error can't find the container with id 3382bbb6e5829428cecc528c12cde2d1a1b003b86ab29325472bbba8a5cd5d38
Jul 30 20:25:23.319125 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:23.319109    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b502fc_da85_472b_947f_0a6e7ca5350b.slice/crio-50160509c454fea704d30ccbed75cf2b42820aaa51e4bfec2c9cb55be3c5cb1e WatchSource:0}: Error finding container 50160509c454fea704d30ccbed75cf2b42820aaa51e4bfec2c9cb55be3c5cb1e: Status 404 returned error can't find the container with id 50160509c454fea704d30ccbed75cf2b42820aaa51e4bfec2c9cb55be3c5cb1e
Jul 30 20:25:23.626063 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.625377    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:23.626063 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.625451    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:23.626063 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.625570    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:23.626063 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.625626    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret podName:643b0bcf-2222-46a0-b8ba-5d556c74e5a3 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:24.625608189 +0000 UTC m=+3.086566996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret") pod "global-pull-secret-syncer-9ttmg" (UID: "643b0bcf-2222-46a0-b8ba-5d556c74e5a3") : object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:23.626063 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.625982    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:23.626063 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.626026    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs podName:d5679c74-1e69-4be8-90d8-3bf6aa168ee4 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:24.626013618 +0000 UTC m=+3.086972410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs") pod "network-metrics-daemon-l6rqg" (UID: "d5679c74-1e69-4be8-90d8-3bf6aa168ee4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:23.726644 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.726031    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:23.726644 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.726202    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 20:25:23.726644 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.726222    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 20:25:23.726644 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.726236    2547 projected.go:196] Error preparing data for projected volume kube-api-access-wpz7j for pod openshift-network-diagnostics/network-check-target-wfrtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:23.726644 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:23.726291    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j podName:b1a23cb4-9909-4e5b-9e7d-79e34f02bb90 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:24.726274217 +0000 UTC m=+3.187233021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wpz7j" (UniqueName: "kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j") pod "network-check-target-wfrtr" (UID: "b1a23cb4-9909-4e5b-9e7d-79e34f02bb90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:23.956578 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.956415    2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 20:20:23 +0000 UTC" deadline="2028-03-10 05:51:06.914433255 +0000 UTC"
Jul 30 20:25:23.956578 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:23.956485    2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14121h25m42.957952631s"
Jul 30 20:25:24.070215 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.069582    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:24.070215 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:24.069707    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:24.099219 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.099181    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sv6zd" event={"ID":"b5b502fc-da85-472b-947f-0a6e7ca5350b","Type":"ContainerStarted","Data":"50160509c454fea704d30ccbed75cf2b42820aaa51e4bfec2c9cb55be3c5cb1e"}
Jul 30 20:25:24.115803 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.115691    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerStarted","Data":"3382bbb6e5829428cecc528c12cde2d1a1b003b86ab29325472bbba8a5cd5d38"}
Jul 30 20:25:24.133380 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.133317    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jvrq2" event={"ID":"6db055c7-8e05-44de-8bc5-716d2b6957f2","Type":"ContainerStarted","Data":"b9904a2865ac75d5d9832e5c8ad62846060a097518a7558e797a2bdff7e14b14"}
Jul 30 20:25:24.145923 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.145889    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8x4qs" event={"ID":"029812bb-2174-49d1-9bc2-9d576b3c2c80","Type":"ContainerStarted","Data":"46a01575270bfa5ecb3b9a4a563a6fa7818839be8b53f0d053c035ba1ef42834"}
Jul 30 20:25:24.156152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.156116    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qbvzb" event={"ID":"9ee03f66-de6a-40c8-a2c0-534ac1754eb2","Type":"ContainerStarted","Data":"6c304d2f15b7e35b7a0f661eb42f52509b1de68d1999ff811a71e219596b8d90"}
Jul 30 20:25:24.165386 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.165357    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8kxwd" event={"ID":"17492ecc-9882-4680-9df3-5ef550df1e0d","Type":"ContainerStarted","Data":"9d12b1b0a585ba52634892ed3c7becb4509e632688932de5633172a16fdc3132"}
Jul 30 20:25:24.181078 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.181020    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dgwck" event={"ID":"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318","Type":"ContainerStarted","Data":"a598b479e2b188fb635baaa97922246d8db3a3c6cac9bb99ee83f6399b4b63d6"}
Jul 30 20:25:24.197980 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.197952    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2s59d" event={"ID":"48cbf098-a8a1-4689-9a3c-ea8784346402","Type":"ContainerStarted","Data":"8f110caa1a054901bbe81ab083b4d63cd571f0d0b5be7cb8e83d886653803702"}
Jul 30 20:25:24.211996 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.211853    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss" event={"ID":"bc16c623-e077-4823-ac5d-d427aa4de728","Type":"ContainerStarted","Data":"bc0eec18070daa2c1bc8561f63097307cfe1bd953623934d3bff94fb9ce1fbca"}
Jul 30 20:25:24.246149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.245874    2547 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 20:25:24.634699 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.634661    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:24.634909 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.634763    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:24.634982 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:24.634930    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:24.635035 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:24.634999    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret podName:643b0bcf-2222-46a0-b8ba-5d556c74e5a3 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:26.63497917 +0000 UTC m=+5.095937974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret") pod "global-pull-secret-syncer-9ttmg" (UID: "643b0bcf-2222-46a0-b8ba-5d556c74e5a3") : object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:24.635504 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:24.635451    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:24.635591 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:24.635515    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs podName:d5679c74-1e69-4be8-90d8-3bf6aa168ee4 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:26.635494163 +0000 UTC m=+5.096452963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs") pod "network-metrics-daemon-l6rqg" (UID: "d5679c74-1e69-4be8-90d8-3bf6aa168ee4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:24.735452 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.735414    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:24.735653 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:24.735636    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 20:25:24.735719 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:24.735662    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 20:25:24.735719 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:24.735677    2547 projected.go:196] Error preparing data for projected volume kube-api-access-wpz7j for pod openshift-network-diagnostics/network-check-target-wfrtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:24.735875 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:24.735759    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j podName:b1a23cb4-9909-4e5b-9e7d-79e34f02bb90 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:26.735716624 +0000 UTC m=+5.196675433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wpz7j" (UniqueName: "kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j") pod "network-check-target-wfrtr" (UID: "b1a23cb4-9909-4e5b-9e7d-79e34f02bb90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:24.958002 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.957907    2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 20:20:23 +0000 UTC" deadline="2028-01-26 21:52:30.737280015 +0000 UTC"
Jul 30 20:25:24.958002 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:24.957955    2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13081h27m5.779333846s"
Jul 30 20:25:25.067125 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:25.067089    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:25.067303 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:25.067222    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:25.067721 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:25.067696    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:25.067868 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:25.067844    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:25.431902 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:25.431876    2547 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 20:25:26.082984 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:26.082913    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:26.083509 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:26.083100    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:26.651537 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:26.651491    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:26.651765 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:26.651579    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:26.651765 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:26.651752    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:26.651883 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:26.651818    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret podName:643b0bcf-2222-46a0-b8ba-5d556c74e5a3 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:30.651798731 +0000 UTC m=+9.112757528 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret") pod "global-pull-secret-syncer-9ttmg" (UID: "643b0bcf-2222-46a0-b8ba-5d556c74e5a3") : object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:26.652301 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:26.652216    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:26.652301 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:26.652271    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs podName:d5679c74-1e69-4be8-90d8-3bf6aa168ee4 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:30.652253156 +0000 UTC m=+9.113211954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs") pod "network-metrics-daemon-l6rqg" (UID: "d5679c74-1e69-4be8-90d8-3bf6aa168ee4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:26.753037 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:26.753004    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:26.753283 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:26.753175    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 20:25:26.753283 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:26.753196    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 20:25:26.753283 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:26.753211    2547 projected.go:196] Error preparing data for projected volume kube-api-access-wpz7j for pod openshift-network-diagnostics/network-check-target-wfrtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:26.753283 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:26.753265    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j podName:b1a23cb4-9909-4e5b-9e7d-79e34f02bb90 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:30.753247823 +0000 UTC m=+9.214206622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wpz7j" (UniqueName: "kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j") pod "network-check-target-wfrtr" (UID: "b1a23cb4-9909-4e5b-9e7d-79e34f02bb90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:27.066892 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:27.066807    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:27.067051 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:27.066989    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:27.067419 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:27.067396    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:27.067521 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:27.067491    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:28.070226 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:28.070197    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:28.070697 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:28.070335    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:29.066469 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:29.066431    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:29.066673 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:29.066562    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:29.067032 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:29.066999    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:29.067159 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:29.067123    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:30.067111 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:30.066571    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:30.067111 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:30.066709    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:30.686362 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:30.686319    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:30.686529 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:30.686399    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:30.686613 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:30.686538    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:30.686613 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:30.686596    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs podName:d5679c74-1e69-4be8-90d8-3bf6aa168ee4 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:38.686576749 +0000 UTC m=+17.147535539 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs") pod "network-metrics-daemon-l6rqg" (UID: "d5679c74-1e69-4be8-90d8-3bf6aa168ee4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:30.686732 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:30.686661    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:30.686732 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:30.686713    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret podName:643b0bcf-2222-46a0-b8ba-5d556c74e5a3 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:38.686684559 +0000 UTC m=+17.147643351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret") pod "global-pull-secret-syncer-9ttmg" (UID: "643b0bcf-2222-46a0-b8ba-5d556c74e5a3") : object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:30.787186 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:30.786854    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:30.787186 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:30.786985    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 20:25:30.787186 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:30.787000    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 20:25:30.787186 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:30.787008    2547 projected.go:196] Error preparing data for projected volume kube-api-access-wpz7j for pod openshift-network-diagnostics/network-check-target-wfrtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:30.787186 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:30.787050    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j podName:b1a23cb4-9909-4e5b-9e7d-79e34f02bb90 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:38.787037618 +0000 UTC m=+17.247996408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wpz7j" (UniqueName: "kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j") pod "network-check-target-wfrtr" (UID: "b1a23cb4-9909-4e5b-9e7d-79e34f02bb90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:31.066846 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:31.066775    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:31.066846 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:31.066809    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:31.067052 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:31.066926    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:31.067106 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:31.067054    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:32.067879 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:32.067852    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:32.068306 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:32.067978    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:33.066518 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:33.066422    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:33.066518 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:33.066474    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:33.066773 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:33.066578    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:33.066773 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:33.066722    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:34.067181 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:34.067148    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:34.067543 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:34.067259    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:35.066672 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:35.066642    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:35.066877 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:35.066648    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:35.066877 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:35.066760    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:35.066877 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:35.066814    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:36.066396 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:36.066328    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:36.066827 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:36.066483    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:37.067115 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:37.066919    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:37.067534 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:37.066919    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:37.067534 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:37.067215    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:37.067534 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:37.067316    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:38.066482 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:38.066450    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:38.066654 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:38.066582    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:38.743247 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:38.743218    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:38.743658 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:38.743268    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:38.743658 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:38.743368    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:38.743658 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:38.743374    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:38.743658 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:38.743427    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret podName:643b0bcf-2222-46a0-b8ba-5d556c74e5a3 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:54.743413616 +0000 UTC m=+33.204372410 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret") pod "global-pull-secret-syncer-9ttmg" (UID: "643b0bcf-2222-46a0-b8ba-5d556c74e5a3") : object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:38.743658 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:38.743441    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs podName:d5679c74-1e69-4be8-90d8-3bf6aa168ee4 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:54.743434767 +0000 UTC m=+33.204393558 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs") pod "network-metrics-daemon-l6rqg" (UID: "d5679c74-1e69-4be8-90d8-3bf6aa168ee4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:38.844495 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:38.844466    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:38.844652 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:38.844633    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 20:25:38.844712 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:38.844660    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 20:25:38.844712 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:38.844672    2547 projected.go:196] Error preparing data for projected volume kube-api-access-wpz7j for pod openshift-network-diagnostics/network-check-target-wfrtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:38.844822 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:38.844742    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j podName:b1a23cb4-9909-4e5b-9e7d-79e34f02bb90 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:54.844726345 +0000 UTC m=+33.305685156 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wpz7j" (UniqueName: "kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j") pod "network-check-target-wfrtr" (UID: "b1a23cb4-9909-4e5b-9e7d-79e34f02bb90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:39.066345 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:39.066274    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:39.066345 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:39.066286    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:39.066557 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:39.066400    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:39.066618 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:39.066559    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:40.067135 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:40.067035    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:40.067509 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:40.067166    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:41.066717 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:41.066670    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:41.066717 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:41.066716    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:41.066915 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:41.066802    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:41.066949 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:41.066929    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:42.066688 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:42.066654    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:42.067155 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:42.066782    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:43.066984 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:43.066948    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:43.067388 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:43.067066    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:43.067388 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:43.067124    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:43.067388 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:43.067247    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:44.066334 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.066310    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:44.066483 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:44.066404    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:44.255136 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.254883    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal" event={"ID":"858f9e0ac50469cf471404be973e16f5","Type":"ContainerStarted","Data":"c2556e04ff45a8af6cbf19393b7de2cb1996772c5b30def58f4c127d640b379a"}
Jul 30 20:25:44.257291 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.257266    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:25:44.257680 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.257656    2547 generic.go:356] "Generic (PLEG): container finished" podID="9bdc8886-b53f-4145-ac66-ac4de098e6e4" containerID="001f8b9199b94f1967f70078e341b17c891c26635e979560ab00433e9eb32f2b" exitCode=1
Jul 30 20:25:44.257792 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.257730    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerStarted","Data":"4cad8a87bd801dd9d954fec1754d7b0da3d317cbc57dd6ee74fffc2b5baadd5c"}
Jul 30 20:25:44.257837 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.257793    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerStarted","Data":"483544ae8079b7979c8699c2671a7bfa3b150dd60b59b2d507fe09af9be89b94"}
Jul 30 20:25:44.257881 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.257848    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerDied","Data":"001f8b9199b94f1967f70078e341b17c891c26635e979560ab00433e9eb32f2b"}
Jul 30 20:25:44.257921 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.257898    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerStarted","Data":"bf5d56ced5d5c263b1b1a772c255da0b719851d90ac27710933a7bdcd11498e3"}
Jul 30 20:25:44.259430 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.259407    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jvrq2" event={"ID":"6db055c7-8e05-44de-8bc5-716d2b6957f2","Type":"ContainerStarted","Data":"43d87ae1803615ff743e28bc81da20e02664d197718d18d4ba4eb0c2fad2ec25"}
Jul 30 20:25:44.262246 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.262224    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dgwck" event={"ID":"39a7b62f-3a6f-4cac-b7ac-7e3d6e735318","Type":"ContainerStarted","Data":"b39b9827e38506bb0217f32ee2a7f35de9dd15f5feb7ace3bd087f2544066cbd"}
Jul 30 20:25:44.267976 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.267883    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-148.ec2.internal" podStartSLOduration=21.267872102 podStartE2EDuration="21.267872102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:44.267828555 +0000 UTC m=+22.728787367" watchObservedRunningTime="2026-07-30 20:25:44.267872102 +0000 UTC m=+22.728830925"
Jul 30 20:25:44.284524 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.284487    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jvrq2" podStartSLOduration=1.599344995 podStartE2EDuration="22.284473555s" totalImagesPullingTime="20.68512856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:44.283985646 +0000 UTC m=+22.744944461" watchObservedRunningTime="2026-07-30 20:25:44.284473555 +0000 UTC m=+22.745432392"
Jul 30 20:25:44.299367 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:44.299334    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dgwck" podStartSLOduration=1.785609252 podStartE2EDuration="22.299320358s" totalImagesPullingTime="20.513711106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:44.299238429 +0000 UTC m=+22.760197241" watchObservedRunningTime="2026-07-30 20:25:44.299320358 +0000 UTC m=+22.760279169"
Jul 30 20:25:45.066104 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.066075    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:45.066228 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:45.066162    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:45.066228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.066085    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:45.066302 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:45.066230    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:45.265443 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.265379    2547 generic.go:356] "Generic (PLEG): container finished" podID="05a45d6bd57d0022f8b25ea0fa9c160f" containerID="aa2479c903e1fa7ea08af535b95c24b274cc5741e219d0877f9555e13a9c2b3b" exitCode=0
Jul 30 20:25:45.266009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.265466    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal" event={"ID":"05a45d6bd57d0022f8b25ea0fa9c160f","Type":"ContainerDied","Data":"aa2479c903e1fa7ea08af535b95c24b274cc5741e219d0877f9555e13a9c2b3b"}
Jul 30 20:25:45.268364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.268343    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:25:45.268734 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.268691    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerStarted","Data":"09c6fc733c5dc4fd6ec816f85d4cbb7d1aafcdec943fd1a9951372b21b5987ca"}
Jul 30 20:25:45.268836 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.268769    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerStarted","Data":"9d4b8a0632b38795d1e5b24fb031e98bc38e93df4d47592125a101a38fdceb46"}
Jul 30 20:25:45.269931 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.269898    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8x4qs" event={"ID":"029812bb-2174-49d1-9bc2-9d576b3c2c80","Type":"ContainerStarted","Data":"c1f99df4fb9c6cd1518ceef7a8a6564473fdc655a4a729112d533517d1f5d26b"}
Jul 30 20:25:45.271151 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.271131    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qbvzb" event={"ID":"9ee03f66-de6a-40c8-a2c0-534ac1754eb2","Type":"ContainerStarted","Data":"99285db6be714dc4c54c4be4e641438824ddf1653a36032f3600879706222af3"}
Jul 30 20:25:45.272445 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.272419    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8kxwd" event={"ID":"17492ecc-9882-4680-9df3-5ef550df1e0d","Type":"ContainerStarted","Data":"4c365cc1658db107598e0c22e8c408b15f8f9609ccb73bc3aef93a57e6ec9c18"}
Jul 30 20:25:45.273879 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.273841    2547 generic.go:356] "Generic (PLEG): container finished" podID="48cbf098-a8a1-4689-9a3c-ea8784346402" containerID="0ce1f87ccc0a7916de80ef37d011ac5600818eac9eeea98a6a0087627f18f939" exitCode=0
Jul 30 20:25:45.273968 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.273908    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2s59d" event={"ID":"48cbf098-a8a1-4689-9a3c-ea8784346402","Type":"ContainerDied","Data":"0ce1f87ccc0a7916de80ef37d011ac5600818eac9eeea98a6a0087627f18f939"}
Jul 30 20:25:45.275365 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.275340    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss" event={"ID":"bc16c623-e077-4823-ac5d-d427aa4de728","Type":"ContainerStarted","Data":"468d365d322a9bab41829cb6167c7732573e9db519def4222bd42828412971be"}
Jul 30 20:25:45.305660 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.305627    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qbvzb" podStartSLOduration=6.9105406259999995 podStartE2EDuration="23.305613664s" totalImagesPullingTime="16.395073038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:45.291960523 +0000 UTC m=+23.752919376" watchObservedRunningTime="2026-07-30 20:25:45.305613664 +0000 UTC m=+23.766572485"
Jul 30 20:25:45.318510 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.318456    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8kxwd" podStartSLOduration=2.871215928 podStartE2EDuration="23.318438441s" totalImagesPullingTime="20.447222513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:45.317917804 +0000 UTC m=+23.778876615" watchObservedRunningTime="2026-07-30 20:25:45.318438441 +0000 UTC m=+23.779397300"
Jul 30 20:25:45.319036 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.319005    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8x4qs" podStartSLOduration=6.9376311489999996 podStartE2EDuration="23.318994248s" totalImagesPullingTime="16.381363099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:45.305292207 +0000 UTC m=+23.766251019" watchObservedRunningTime="2026-07-30 20:25:45.318994248 +0000 UTC m=+23.779953062"
Jul 30 20:25:45.928846 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.928813    2547 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 20:25:45.984827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.984719    2547 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T20:25:45.928828313Z","UUID":"aa77406a-7bf8-4a0a-9142-95f942c257d2","Handler":null,"Name":"","Endpoint":""}
Jul 30 20:25:45.986016 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.986000    2547 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 20:25:45.986105 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:45.986038    2547 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 20:25:46.066893 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:46.066835    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:46.067017 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:46.066930    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:46.279262 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:46.279227    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss" event={"ID":"bc16c623-e077-4823-ac5d-d427aa4de728","Type":"ContainerStarted","Data":"6ddeaae20f4e5916d3a404fecec37178ad4e9c9e76f4a9e05937521059c6ee91"}
Jul 30 20:25:46.280958 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:46.280914    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal" event={"ID":"05a45d6bd57d0022f8b25ea0fa9c160f","Type":"ContainerStarted","Data":"b9de5e7c132b0a57b954eccea9996876b4bf22a35c484d76e4b80b102f591277"}
Jul 30 20:25:46.282335 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:46.282287    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sv6zd" event={"ID":"b5b502fc-da85-472b-947f-0a6e7ca5350b","Type":"ContainerStarted","Data":"c98ae39a5630f4cc4d2044b44c6c59da3daff15cc4b66ffc2f6caaf3af7e2ba2"}
Jul 30 20:25:46.297130 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:46.297090    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-148.ec2.internal" podStartSLOduration=23.29707863 podStartE2EDuration="23.29707863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:46.29657828 +0000 UTC m=+24.757537091" watchObservedRunningTime="2026-07-30 20:25:46.29707863 +0000 UTC m=+24.758037441"
Jul 30 20:25:46.988898 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:46.988863    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:46.989549 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:46.989518    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:47.006515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:47.006474    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sv6zd" podStartSLOduration=3.5141624 podStartE2EDuration="24.006460592s" totalImagesPullingTime="20.492298192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:46.309394498 +0000 UTC m=+24.770353321" watchObservedRunningTime="2026-07-30 20:25:47.006460592 +0000 UTC m=+25.467419403"
Jul 30 20:25:47.066823 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:47.066795    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:47.066942 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:47.066800    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:47.066942 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:47.066902    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:47.067043 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:47.067018    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:47.289884 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:47.289821    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:25:47.290344 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:47.290319    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerStarted","Data":"80a535999ee23137f0e883e959f309de77d5b4f7c6d4a629364c58f1510a2133"}
Jul 30 20:25:47.290905 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:47.290883    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:47.291049 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:47.290960    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qbvzb"
Jul 30 20:25:48.066626 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:48.066595    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:48.066826 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:48.066749    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:48.295158 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:48.295123    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss" event={"ID":"bc16c623-e077-4823-ac5d-d427aa4de728","Type":"ContainerStarted","Data":"d9df9698530d7de2e21c94d78f23983f86dd01923547ae092bad4e93785fb7a4"}
Jul 30 20:25:48.313809 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:48.313761    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d7nss" podStartSLOduration=3.205592847 podStartE2EDuration="26.313743296s" totalImagesPullingTime="23.108150449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:48.313597409 +0000 UTC m=+26.774556222" watchObservedRunningTime="2026-07-30 20:25:48.313743296 +0000 UTC m=+26.774702126"
Jul 30 20:25:49.066962 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:49.066692    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:49.067097 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:49.066758    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:49.067097 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:49.067019    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:49.067203 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:49.067096    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:49.301381 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:49.301352    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:25:49.301999 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:49.301822    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerStarted","Data":"74d58dede1df6aa577f2c9835ed26e3ddf89639ca39c31609c7d3cbe93d6d78a"}
Jul 30 20:25:49.302324 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:49.302303    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:49.302451 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:49.302371    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:49.302451 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:49.302432    2547 scope.go:117] "RemoveContainer" containerID="001f8b9199b94f1967f70078e341b17c891c26635e979560ab00433e9eb32f2b"
Jul 30 20:25:49.320066 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:49.319970    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:49.320561 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:49.320507    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:50.068740 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:50.068506    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:50.068740 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:50.068637    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:51.010731 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.010441    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wfrtr"]
Jul 30 20:25:51.011328 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.010744    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:51.011328 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:51.010833    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:51.014431 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.014408    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9ttmg"]
Jul 30 20:25:51.014554 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.014495    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:51.014615 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:51.014575    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:51.015226 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.015207    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l6rqg"]
Jul 30 20:25:51.015327 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.015309    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:51.015447 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:51.015427    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:51.310364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.310270    2547 generic.go:356] "Generic (PLEG): container finished" podID="48cbf098-a8a1-4689-9a3c-ea8784346402" containerID="7d4951ba73300d9052a53eaade7a6c0662debf2b0893ed069262a735c692dd24" exitCode=0
Jul 30 20:25:51.310566 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.310360    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2s59d" event={"ID":"48cbf098-a8a1-4689-9a3c-ea8784346402","Type":"ContainerDied","Data":"7d4951ba73300d9052a53eaade7a6c0662debf2b0893ed069262a735c692dd24"}
Jul 30 20:25:51.314317 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.314300    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:25:51.314670 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.314648    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" event={"ID":"9bdc8886-b53f-4145-ac66-ac4de098e6e4","Type":"ContainerStarted","Data":"8e6e4aa26970062f75cf04e3c024cdec061cc51375f0d5dc42ad6c60e6dac4fa"}
Jul 30 20:25:51.314771 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.314759    2547 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 30 20:25:51.363715 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.363657    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" podStartSLOduration=8.818135573 podStartE2EDuration="29.363642435s" totalImagesPullingTime="20.545506862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:51.363626888 +0000 UTC m=+29.824585700" watchObservedRunningTime="2026-07-30 20:25:51.363642435 +0000 UTC m=+29.824601241"
Jul 30 20:25:51.966137 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:51.966102    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:25:52.066772 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:52.066748    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:52.067143 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:52.066873    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:52.318622 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:52.318543    2547 generic.go:356] "Generic (PLEG): container finished" podID="48cbf098-a8a1-4689-9a3c-ea8784346402" containerID="97bf882e6e9245cec081331a9a592ec3dbdfd27fdaca750158ba3e9727052aa7" exitCode=0
Jul 30 20:25:52.318795 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:52.318622    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2s59d" event={"ID":"48cbf098-a8a1-4689-9a3c-ea8784346402","Type":"ContainerDied","Data":"97bf882e6e9245cec081331a9a592ec3dbdfd27fdaca750158ba3e9727052aa7"}
Jul 30 20:25:53.066539 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:53.066464    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:53.066539 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:53.066502    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:53.066775 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:53.066595    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:53.066775 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:53.066750    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:53.323744 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:53.323681    2547 generic.go:356] "Generic (PLEG): container finished" podID="48cbf098-a8a1-4689-9a3c-ea8784346402" containerID="eb2fbb3c4a12d8497a44e3bb596c6fa470b952226571b8977c25ee91f13ec3f6" exitCode=0
Jul 30 20:25:53.323929 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:53.323744    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2s59d" event={"ID":"48cbf098-a8a1-4689-9a3c-ea8784346402","Type":"ContainerDied","Data":"eb2fbb3c4a12d8497a44e3bb596c6fa470b952226571b8977c25ee91f13ec3f6"}
Jul 30 20:25:53.334500 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:53.334464    2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr" podUID="9bdc8886-b53f-4145-ac66-ac4de098e6e4" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 30 20:25:54.066892 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:54.066863    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:54.067355 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:54.067004    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:54.764912 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:54.764880    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:54.765069 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:54.764941    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:54.765069 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:54.765050    2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:54.765148 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:54.765050    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:54.765148 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:54.765116    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs podName:d5679c74-1e69-4be8-90d8-3bf6aa168ee4 nodeName:}" failed. No retries permitted until 2026-07-30 20:26:26.765097177 +0000 UTC m=+65.226055979 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs") pod "network-metrics-daemon-l6rqg" (UID: "d5679c74-1e69-4be8-90d8-3bf6aa168ee4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 20:25:54.765148 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:54.765133    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret podName:643b0bcf-2222-46a0-b8ba-5d556c74e5a3 nodeName:}" failed. No retries permitted until 2026-07-30 20:26:26.765124815 +0000 UTC m=+65.226083606 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret") pod "global-pull-secret-syncer-9ttmg" (UID: "643b0bcf-2222-46a0-b8ba-5d556c74e5a3") : object "kube-system"/"original-pull-secret" not registered
Jul 30 20:25:54.865889 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:54.865859    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:54.866076 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:54.866005    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 20:25:54.866076 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:54.866023    2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 20:25:54.866076 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:54.866036    2547 projected.go:196] Error preparing data for projected volume kube-api-access-wpz7j for pod openshift-network-diagnostics/network-check-target-wfrtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:54.866224 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:54.866094    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j podName:b1a23cb4-9909-4e5b-9e7d-79e34f02bb90 nodeName:}" failed. No retries permitted until 2026-07-30 20:26:26.866074539 +0000 UTC m=+65.327033344 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wpz7j" (UniqueName: "kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j") pod "network-check-target-wfrtr" (UID: "b1a23cb4-9909-4e5b-9e7d-79e34f02bb90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 20:25:55.066426 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:55.066336    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:55.066594 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:55.066459    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfrtr" podUID="b1a23cb4-9909-4e5b-9e7d-79e34f02bb90"
Jul 30 20:25:55.066594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:55.066487    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:55.066723 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:55.066603    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ttmg" podUID="643b0bcf-2222-46a0-b8ba-5d556c74e5a3"
Jul 30 20:25:56.067042 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:56.067005    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:56.067455 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:56.067159    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:25:56.886840 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:56.886611    2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-148.ec2.internal" event="NodeReady"
Jul 30 20:25:56.887121 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:56.886995    2547 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 20:25:56.929827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:56.929798    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tjtsk"]
Jul 30 20:25:56.991834 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:56.991797    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-48ck8"]
Jul 30 20:25:56.992137 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:56.992110    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:56.994736 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:56.994692    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 20:25:56.994862 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:56.994794    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 20:25:56.994934 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:56.994794    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tn7n7\""
Jul 30 20:25:57.004384 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.004367    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tjtsk"]
Jul 30 20:25:57.004496 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.004398    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-48ck8"]
Jul 30 20:25:57.004496 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.004459    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-48ck8"
Jul 30 20:25:57.009808 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.009400    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 20:25:57.009808 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.009520    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 20:25:57.009808 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.009678    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-798xg\""
Jul 30 20:25:57.009983 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.009970    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 20:25:57.066749 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.066719    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:25:57.066912 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.066765    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:25:57.069831 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.069808    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-944cj\""
Jul 30 20:25:57.070249 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.069808    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 20:25:57.070249 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.069810    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 20:25:57.070249 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.069810    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 20:25:57.080305 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.080279    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l78h\" (UniqueName: \"kubernetes.io/projected/ae70ec57-f212-4922-8ea3-c2704e36af32-kube-api-access-6l78h\") pod \"ingress-canary-48ck8\" (UID: \"ae70ec57-f212-4922-8ea3-c2704e36af32\") " pod="openshift-ingress-canary/ingress-canary-48ck8"
Jul 30 20:25:57.080419 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.080329    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.080419 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.080380    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdmfr\" (UniqueName: \"kubernetes.io/projected/4bfaea5d-a4b2-42a0-889d-29b157f75146-kube-api-access-mdmfr\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.080516 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.080479    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae70ec57-f212-4922-8ea3-c2704e36af32-cert\") pod \"ingress-canary-48ck8\" (UID: \"ae70ec57-f212-4922-8ea3-c2704e36af32\") " pod="openshift-ingress-canary/ingress-canary-48ck8"
Jul 30 20:25:57.080516 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.080510    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4bfaea5d-a4b2-42a0-889d-29b157f75146-config-volume\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.080602 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.080534    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bfaea5d-a4b2-42a0-889d-29b157f75146-tmp-dir\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.181287 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.181190    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae70ec57-f212-4922-8ea3-c2704e36af32-cert\") pod \"ingress-canary-48ck8\" (UID: \"ae70ec57-f212-4922-8ea3-c2704e36af32\") " pod="openshift-ingress-canary/ingress-canary-48ck8"
Jul 30 20:25:57.181287 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.181246    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4bfaea5d-a4b2-42a0-889d-29b157f75146-config-volume\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.181509 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.181463    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bfaea5d-a4b2-42a0-889d-29b157f75146-tmp-dir\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.181568 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.181517    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l78h\" (UniqueName: \"kubernetes.io/projected/ae70ec57-f212-4922-8ea3-c2704e36af32-kube-api-access-6l78h\") pod \"ingress-canary-48ck8\" (UID: \"ae70ec57-f212-4922-8ea3-c2704e36af32\") " pod="openshift-ingress-canary/ingress-canary-48ck8"
Jul 30 20:25:57.181568 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.181554    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.181678 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.181587    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdmfr\" (UniqueName: \"kubernetes.io/projected/4bfaea5d-a4b2-42a0-889d-29b157f75146-kube-api-access-mdmfr\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.181829 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.181807    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bfaea5d-a4b2-42a0-889d-29b157f75146-tmp-dir\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.181829 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:57.181823    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 20:25:57.181932 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.181868    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4bfaea5d-a4b2-42a0-889d-29b157f75146-config-volume\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.181932 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:57.181876    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls podName:4bfaea5d-a4b2-42a0-889d-29b157f75146 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:57.681859624 +0000 UTC m=+36.142818422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls") pod "dns-default-tjtsk" (UID: "4bfaea5d-a4b2-42a0-889d-29b157f75146") : secret "dns-default-metrics-tls" not found
Jul 30 20:25:57.186944 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.186916    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae70ec57-f212-4922-8ea3-c2704e36af32-cert\") pod \"ingress-canary-48ck8\" (UID: \"ae70ec57-f212-4922-8ea3-c2704e36af32\") " pod="openshift-ingress-canary/ingress-canary-48ck8"
Jul 30 20:25:57.193283 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.193229    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdmfr\" (UniqueName: \"kubernetes.io/projected/4bfaea5d-a4b2-42a0-889d-29b157f75146-kube-api-access-mdmfr\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.193406 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.193366    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l78h\" (UniqueName: \"kubernetes.io/projected/ae70ec57-f212-4922-8ea3-c2704e36af32-kube-api-access-6l78h\") pod \"ingress-canary-48ck8\" (UID: \"ae70ec57-f212-4922-8ea3-c2704e36af32\") " pod="openshift-ingress-canary/ingress-canary-48ck8"
Jul 30 20:25:57.313668 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.313629    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-48ck8"
Jul 30 20:25:57.501171 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.501141    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-48ck8"]
Jul 30 20:25:57.540061 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:25:57.540027    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae70ec57_f212_4922_8ea3_c2704e36af32.slice/crio-ba2cb6828e69f5ece56cb8607e1da7cf145d9cc70e8b4734c55243653fc4ba0a WatchSource:0}: Error finding container ba2cb6828e69f5ece56cb8607e1da7cf145d9cc70e8b4734c55243653fc4ba0a: Status 404 returned error can't find the container with id ba2cb6828e69f5ece56cb8607e1da7cf145d9cc70e8b4734c55243653fc4ba0a
Jul 30 20:25:57.685852 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:57.685817    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:57.685995 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:57.685968    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 20:25:57.686058 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:57.686038    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls podName:4bfaea5d-a4b2-42a0-889d-29b157f75146 nodeName:}" failed. No retries permitted until 2026-07-30 20:25:58.686016843 +0000 UTC m=+37.146975650 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls") pod "dns-default-tjtsk" (UID: "4bfaea5d-a4b2-42a0-889d-29b157f75146") : secret "dns-default-metrics-tls" not found
Jul 30 20:25:58.070336 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:58.070091    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:25:58.074790 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:58.074602    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4nz9n\""
Jul 30 20:25:58.075211 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:58.075189    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 20:25:58.340291 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:58.340161    2547 generic.go:356] "Generic (PLEG): container finished" podID="48cbf098-a8a1-4689-9a3c-ea8784346402" containerID="cbb73a10f43e9ff01bf0e75d41eeabad4bfc9285f3445bd70060cf3580113e07" exitCode=0
Jul 30 20:25:58.340291 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:58.340245    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2s59d" event={"ID":"48cbf098-a8a1-4689-9a3c-ea8784346402","Type":"ContainerDied","Data":"cbb73a10f43e9ff01bf0e75d41eeabad4bfc9285f3445bd70060cf3580113e07"}
Jul 30 20:25:58.341493 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:58.341442    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-48ck8" event={"ID":"ae70ec57-f212-4922-8ea3-c2704e36af32","Type":"ContainerStarted","Data":"ba2cb6828e69f5ece56cb8607e1da7cf145d9cc70e8b4734c55243653fc4ba0a"}
Jul 30 20:25:58.694357 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:58.694262    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:25:58.694510 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:58.694405    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 20:25:58.694510 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:25:58.694475    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls podName:4bfaea5d-a4b2-42a0-889d-29b157f75146 nodeName:}" failed. No retries permitted until 2026-07-30 20:26:00.694455394 +0000 UTC m=+39.155414198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls") pod "dns-default-tjtsk" (UID: "4bfaea5d-a4b2-42a0-889d-29b157f75146") : secret "dns-default-metrics-tls" not found
Jul 30 20:25:59.347270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:59.347234    2547 generic.go:356] "Generic (PLEG): container finished" podID="48cbf098-a8a1-4689-9a3c-ea8784346402" containerID="3f55f035cafcd89a80539b7bf835febf4f1e7918ea55adca1e4390363d024b58" exitCode=0
Jul 30 20:25:59.347750 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:25:59.347313    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2s59d" event={"ID":"48cbf098-a8a1-4689-9a3c-ea8784346402","Type":"ContainerDied","Data":"3f55f035cafcd89a80539b7bf835febf4f1e7918ea55adca1e4390363d024b58"}
Jul 30 20:26:00.353604 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:00.353569    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2s59d" event={"ID":"48cbf098-a8a1-4689-9a3c-ea8784346402","Type":"ContainerStarted","Data":"72b2a3f36961b4f15bebc6da6f6ebe8cfb533e89d829e8842ea23e624da3f9b5"}
Jul 30 20:26:00.355267 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:00.355243    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-48ck8" event={"ID":"ae70ec57-f212-4922-8ea3-c2704e36af32","Type":"ContainerStarted","Data":"057eb13f1aa57654f91c19ee100a988c39d1c974993a136724f6de93fc9f0afa"}
Jul 30 20:26:00.376448 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:00.376396    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2s59d" podStartSLOduration=6.924455669 podStartE2EDuration="38.376381337s" totalImagesPullingTime="31.451925668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:26:00.375081721 +0000 UTC m=+38.836040533" watchObservedRunningTime="2026-07-30 20:26:00.376381337 +0000 UTC m=+38.837340150"
Jul 30 20:26:00.394778 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:00.394731    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-48ck8" podStartSLOduration=1.79538848 podStartE2EDuration="4.394715314s" totalImagesPullingTime="2.599326834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:26:00.393942229 +0000 UTC m=+38.854901052" watchObservedRunningTime="2026-07-30 20:26:00.394715314 +0000 UTC m=+38.855674117"
Jul 30 20:26:00.711806 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:00.711765    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:26:00.711971 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:00.711881    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 20:26:00.711971 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:00.711933    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls podName:4bfaea5d-a4b2-42a0-889d-29b157f75146 nodeName:}" failed. No retries permitted until 2026-07-30 20:26:04.711920406 +0000 UTC m=+43.172879201 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls") pod "dns-default-tjtsk" (UID: "4bfaea5d-a4b2-42a0-889d-29b157f75146") : secret "dns-default-metrics-tls" not found
Jul 30 20:26:04.739149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:04.739098    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:26:04.739549 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:04.739277    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 20:26:04.739549 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:04.739356    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls podName:4bfaea5d-a4b2-42a0-889d-29b157f75146 nodeName:}" failed. No retries permitted until 2026-07-30 20:26:12.739338898 +0000 UTC m=+51.200297689 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls") pod "dns-default-tjtsk" (UID: "4bfaea5d-a4b2-42a0-889d-29b157f75146") : secret "dns-default-metrics-tls" not found
Jul 30 20:26:12.800064 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:12.800013    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:26:12.800544 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:12.800171    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 20:26:12.800544 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:12.800257    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls podName:4bfaea5d-a4b2-42a0-889d-29b157f75146 nodeName:}" failed. No retries permitted until 2026-07-30 20:26:28.800238189 +0000 UTC m=+67.261196980 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls") pod "dns-default-tjtsk" (UID: "4bfaea5d-a4b2-42a0-889d-29b157f75146") : secret "dns-default-metrics-tls" not found
Jul 30 20:26:16.698951 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.698913    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"]
Jul 30 20:26:16.700925 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.700898    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:16.703636 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.703604    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 20:26:16.704957 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.704930    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 30 20:26:16.704957 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.704934    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 20:26:16.705118 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.704962    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 20:26:16.711654 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.711627    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"]
Jul 30 20:26:16.724173 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.724137    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/141aceec-40d2-416e-b90d-fb9b8c24d9d7-klusterlet-config\") pod \"klusterlet-addon-workmgr-b5c9474f8-lmsv8\" (UID: \"141aceec-40d2-416e-b90d-fb9b8c24d9d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:16.724173 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.724176    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wvtd\" (UniqueName: \"kubernetes.io/projected/141aceec-40d2-416e-b90d-fb9b8c24d9d7-kube-api-access-6wvtd\") pod \"klusterlet-addon-workmgr-b5c9474f8-lmsv8\" (UID: \"141aceec-40d2-416e-b90d-fb9b8c24d9d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:16.724415 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.724209    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/141aceec-40d2-416e-b90d-fb9b8c24d9d7-tmp\") pod \"klusterlet-addon-workmgr-b5c9474f8-lmsv8\" (UID: \"141aceec-40d2-416e-b90d-fb9b8c24d9d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:16.824797 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.824754    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/141aceec-40d2-416e-b90d-fb9b8c24d9d7-klusterlet-config\") pod \"klusterlet-addon-workmgr-b5c9474f8-lmsv8\" (UID: \"141aceec-40d2-416e-b90d-fb9b8c24d9d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:16.825009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.824803    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wvtd\" (UniqueName: \"kubernetes.io/projected/141aceec-40d2-416e-b90d-fb9b8c24d9d7-kube-api-access-6wvtd\") pod \"klusterlet-addon-workmgr-b5c9474f8-lmsv8\" (UID: \"141aceec-40d2-416e-b90d-fb9b8c24d9d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:16.825009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.824847    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/141aceec-40d2-416e-b90d-fb9b8c24d9d7-tmp\") pod \"klusterlet-addon-workmgr-b5c9474f8-lmsv8\" (UID: \"141aceec-40d2-416e-b90d-fb9b8c24d9d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:16.825264 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.825242    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/141aceec-40d2-416e-b90d-fb9b8c24d9d7-tmp\") pod \"klusterlet-addon-workmgr-b5c9474f8-lmsv8\" (UID: \"141aceec-40d2-416e-b90d-fb9b8c24d9d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:16.829182 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.829155    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/141aceec-40d2-416e-b90d-fb9b8c24d9d7-klusterlet-config\") pod \"klusterlet-addon-workmgr-b5c9474f8-lmsv8\" (UID: \"141aceec-40d2-416e-b90d-fb9b8c24d9d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:16.833717 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:16.833671    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wvtd\" (UniqueName: \"kubernetes.io/projected/141aceec-40d2-416e-b90d-fb9b8c24d9d7-kube-api-access-6wvtd\") pod \"klusterlet-addon-workmgr-b5c9474f8-lmsv8\" (UID: \"141aceec-40d2-416e-b90d-fb9b8c24d9d7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:17.011859 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:17.011766    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:17.142375 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:17.142345    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"]
Jul 30 20:26:17.145550 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:26:17.145518    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141aceec_40d2_416e_b90d_fb9b8c24d9d7.slice/crio-b5f94946acb836f73378a75d1f8f00af568cd006687b6030dd06e4849a3cfc1c WatchSource:0}: Error finding container b5f94946acb836f73378a75d1f8f00af568cd006687b6030dd06e4849a3cfc1c: Status 404 returned error can't find the container with id b5f94946acb836f73378a75d1f8f00af568cd006687b6030dd06e4849a3cfc1c
Jul 30 20:26:17.398346 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:17.398311    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8" event={"ID":"141aceec-40d2-416e-b90d-fb9b8c24d9d7","Type":"ContainerStarted","Data":"b5f94946acb836f73378a75d1f8f00af568cd006687b6030dd06e4849a3cfc1c"}
Jul 30 20:26:22.411824 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:22.411786    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8" event={"ID":"141aceec-40d2-416e-b90d-fb9b8c24d9d7","Type":"ContainerStarted","Data":"625e1328d186191001a60b0242e5cfcba970070205231d9e40f70940f635be42"}
Jul 30 20:26:22.412205 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:22.412003    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:22.413487 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:22.413467    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:26:22.428541 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:22.428489    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8" podStartSLOduration=2.045131304 podStartE2EDuration="6.428473968s" totalImagesPullingTime="4.383342664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:26:22.428003121 +0000 UTC m=+60.888961933" watchObservedRunningTime="2026-07-30 20:26:22.428473968 +0000 UTC m=+60.889432781"
Jul 30 20:26:23.336067 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:23.336043    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-clxfr"
Jul 30 20:26:26.785563 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:26.785512    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:26:26.786032 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:26.785604    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:26:26.788240 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:26.788215    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 20:26:26.788396 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:26.788378    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 20:26:26.796460 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:26.796429    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 20:26:26.796537 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:26.796524    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs podName:d5679c74-1e69-4be8-90d8-3bf6aa168ee4 nodeName:}" failed. No retries permitted until 2026-07-30 20:27:30.796502627 +0000 UTC m=+129.257461421 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs") pod "network-metrics-daemon-l6rqg" (UID: "d5679c74-1e69-4be8-90d8-3bf6aa168ee4") : secret "metrics-daemon-secret" not found
Jul 30 20:26:26.798914 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:26.798882    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/643b0bcf-2222-46a0-b8ba-5d556c74e5a3-original-pull-secret\") pod \"global-pull-secret-syncer-9ttmg\" (UID: \"643b0bcf-2222-46a0-b8ba-5d556c74e5a3\") " pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:26:26.886775 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:26.886735    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:26:26.889793 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:26.889743    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 20:26:26.899793 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:26.899765    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 20:26:26.910746 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:26.910716    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpz7j\" (UniqueName: \"kubernetes.io/projected/b1a23cb4-9909-4e5b-9e7d-79e34f02bb90-kube-api-access-wpz7j\") pod \"network-check-target-wfrtr\" (UID: \"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90\") " pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:26:27.078186 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:27.078153    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ttmg"
Jul 30 20:26:27.086111 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:27.086085    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-944cj\""
Jul 30 20:26:27.094424 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:27.094389    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:26:27.244544 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:27.244508    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9ttmg"]
Jul 30 20:26:27.262934 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:27.262894    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wfrtr"]
Jul 30 20:26:27.266254 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:26:27.266226    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a23cb4_9909_4e5b_9e7d_79e34f02bb90.slice/crio-769003eee20aef503cd2daaae0c59326bb50427bc8f4aabc8dcad30192c9cb2b WatchSource:0}: Error finding container 769003eee20aef503cd2daaae0c59326bb50427bc8f4aabc8dcad30192c9cb2b: Status 404 returned error can't find the container with id 769003eee20aef503cd2daaae0c59326bb50427bc8f4aabc8dcad30192c9cb2b
Jul 30 20:26:27.425021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:27.424930    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wfrtr" event={"ID":"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90","Type":"ContainerStarted","Data":"769003eee20aef503cd2daaae0c59326bb50427bc8f4aabc8dcad30192c9cb2b"}
Jul 30 20:26:27.426028 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:27.425989    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9ttmg" event={"ID":"643b0bcf-2222-46a0-b8ba-5d556c74e5a3","Type":"ContainerStarted","Data":"0fe0b53306c8df19fe29e0943550cbcd25c716d17d37b975e288975f87504550"}
Jul 30 20:26:28.801461 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:28.801418    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:26:28.801917 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:28.801555    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 20:26:28.801917 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:26:28.801625    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls podName:4bfaea5d-a4b2-42a0-889d-29b157f75146 nodeName:}" failed. No retries permitted until 2026-07-30 20:27:00.801605243 +0000 UTC m=+99.262564038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls") pod "dns-default-tjtsk" (UID: "4bfaea5d-a4b2-42a0-889d-29b157f75146") : secret "dns-default-metrics-tls" not found
Jul 30 20:26:32.442851 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:32.442789    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wfrtr" event={"ID":"b1a23cb4-9909-4e5b-9e7d-79e34f02bb90","Type":"ContainerStarted","Data":"67e102649e0142f45b7f7385578e4dce43196b3a0981edee9fc749378e1ea81b"}
Jul 30 20:26:32.443333 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:32.442865    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:26:32.444494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:32.444465    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9ttmg" event={"ID":"643b0bcf-2222-46a0-b8ba-5d556c74e5a3","Type":"ContainerStarted","Data":"2e1a8ed374f1180307def84ea243cd63407babcf5e930d48eea167291e2871c4"}
Jul 30 20:26:32.459555 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:32.459498    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wfrtr" podStartSLOduration=65.508017937 podStartE2EDuration="1m10.45948231s" totalImagesPullingTime="4.951464373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:26:32.458199647 +0000 UTC m=+70.919158460" watchObservedRunningTime="2026-07-30 20:26:32.45948231 +0000 UTC m=+70.920441120"
Jul 30 20:26:32.475974 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:26:32.475865    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9ttmg" podStartSLOduration=65.503620716 podStartE2EDuration="1m10.475845213s" totalImagesPullingTime="4.972224497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:26:32.474837654 +0000 UTC m=+70.935796464" watchObservedRunningTime="2026-07-30 20:26:32.475845213 +0000 UTC m=+70.936804024"
Jul 30 20:27:00.813391 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:27:00.813291    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:27:00.813834 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:27:00.813460    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 20:27:00.813834 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:27:00.813550    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls podName:4bfaea5d-a4b2-42a0-889d-29b157f75146 nodeName:}" failed. No retries permitted until 2026-07-30 20:28:04.813529156 +0000 UTC m=+163.274487950 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls") pod "dns-default-tjtsk" (UID: "4bfaea5d-a4b2-42a0-889d-29b157f75146") : secret "dns-default-metrics-tls" not found
Jul 30 20:27:03.450118 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:27:03.450076    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wfrtr"
Jul 30 20:27:30.803347 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:27:30.803296    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:27:30.803919 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:27:30.803459    2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 20:27:30.803919 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:27:30.803536    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs podName:d5679c74-1e69-4be8-90d8-3bf6aa168ee4 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:32.803516622 +0000 UTC m=+251.264475412 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs") pod "network-metrics-daemon-l6rqg" (UID: "d5679c74-1e69-4be8-90d8-3bf6aa168ee4") : secret "metrics-daemon-secret" not found
Jul 30 20:28:00.003416 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:28:00.003361    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tjtsk" podUID="4bfaea5d-a4b2-42a0-889d-29b157f75146"
Jul 30 20:28:00.658498 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:00.658473    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:28:01.082487 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:28:01.082452    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-l6rqg" podUID="d5679c74-1e69-4be8-90d8-3bf6aa168ee4"
Jul 30 20:28:04.824368 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:04.824318    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:28:04.824878 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:28:04.824463    2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 20:28:04.824878 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:28:04.824517    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls podName:4bfaea5d-a4b2-42a0-889d-29b157f75146 nodeName:}" failed. No retries permitted until 2026-07-30 20:30:06.824503372 +0000 UTC m=+285.285462165 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls") pod "dns-default-tjtsk" (UID: "4bfaea5d-a4b2-42a0-889d-29b157f75146") : secret "dns-default-metrics-tls" not found
Jul 30 20:28:15.066894 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:15.066806    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:28:21.706306 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:21.706274    2547 generic.go:356] "Generic (PLEG): container finished" podID="141aceec-40d2-416e-b90d-fb9b8c24d9d7" containerID="625e1328d186191001a60b0242e5cfcba970070205231d9e40f70940f635be42" exitCode=1
Jul 30 20:28:21.706671 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:21.706350    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8" event={"ID":"141aceec-40d2-416e-b90d-fb9b8c24d9d7","Type":"ContainerDied","Data":"625e1328d186191001a60b0242e5cfcba970070205231d9e40f70940f635be42"}
Jul 30 20:28:21.706737 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:21.706676    2547 scope.go:117] "RemoveContainer" containerID="625e1328d186191001a60b0242e5cfcba970070205231d9e40f70940f635be42"
Jul 30 20:28:22.412491 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:22.412435    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:28:22.712766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:22.712660    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8" event={"ID":"141aceec-40d2-416e-b90d-fb9b8c24d9d7","Type":"ContainerStarted","Data":"9cac42ba9f59443433a84b41614e0ece8404a27515936d1093f6bf93d20d9eb5"}
Jul 30 20:28:22.713149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:22.712936    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:28:22.713547 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:28:22.713529    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b5c9474f8-lmsv8"
Jul 30 20:29:08.874997 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.874958    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-7xmcm"]
Jul 30 20:29:08.876732 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.876714    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75757fc68b-5gs8h"]
Jul 30 20:29:08.876831 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.876813    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:08.878588 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.878570    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:08.879289 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.879271    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 30 20:29:08.879614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.879596    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 30 20:29:08.879884 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.879867    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 30 20:29:08.880047 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.880027    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 30 20:29:08.880120 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.880073    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-667zl\""
Jul 30 20:29:08.881231 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.881205    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 20:29:08.881428 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.881413    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 20:29:08.881609 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.881597    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mzftn\""
Jul 30 20:29:08.885246 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.885215    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 20:29:08.888026 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.888009    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 30 20:29:08.889666 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.889648    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 20:29:08.900715 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.900673    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-7xmcm"]
Jul 30 20:29:08.905824 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.905801    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75757fc68b-5gs8h"]
Jul 30 20:29:08.953138 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953102    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-image-registry-private-configuration\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:08.953138 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953139    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fe3690a-68d4-4e16-9871-c258935edb0e-serving-cert\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:08.953364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953171    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e49f3bf5-4164-4228-bdb2-5145951d33e0-ca-trust-extracted\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:08.953364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953219    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tpq2\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-kube-api-access-9tpq2\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:08.953364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953247    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-trusted-ca\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:08.953364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953268    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-installation-pull-secrets\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:08.953364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953284    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8fe3690a-68d4-4e16-9871-c258935edb0e-config\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:08.953364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953303    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8fe3690a-68d4-4e16-9871-c258935edb0e-trusted-ca\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:08.953364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953319    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r4t8\" (UniqueName: \"kubernetes.io/projected/8fe3690a-68d4-4e16-9871-c258935edb0e-kube-api-access-5r4t8\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:08.953581 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953388    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:08.953581 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953405    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-bound-sa-token\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:08.953581 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.953435    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-certificates\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:08.970761 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.970736    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"]
Jul 30 20:29:08.972798 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.972783    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68cfb99b6d-9cgw6"]
Jul 30 20:29:08.972915 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.972900    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:08.974505 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.974489    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:08.975307 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.975290    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ftt9p\""
Jul 30 20:29:08.975442 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.975426    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 30 20:29:08.975525 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.975473    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 30 20:29:08.975632 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.975621    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 30 20:29:08.978053 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.978035    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 30 20:29:08.978155 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.978119    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 30 20:29:08.978155 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.978062    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 30 20:29:08.978266 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.978173    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 20:29:08.978266 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.978186    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2nwg9\""
Jul 30 20:29:08.978378 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.978294    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 20:29:08.978509 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.978494    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 30 20:29:08.986928 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.986908    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"]
Jul 30 20:29:08.988180 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:08.988161    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68cfb99b6d-9cgw6"]
Jul 30 20:29:09.053730 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.053682    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:09.053918 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.053736    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gjmp\" (UniqueName: \"kubernetes.io/projected/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-kube-api-access-5gjmp\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.053918 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.053756    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hschs\" (UniqueName: \"kubernetes.io/projected/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-kube-api-access-hschs\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:09.053918 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.053786    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-certificates\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.053918 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.053804    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.053918 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.053820    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fe3690a-68d4-4e16-9871-c258935edb0e-serving-cert\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:09.053918 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.053835    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-default-certificate\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.053918 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.053849    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.054228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.053971    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r4t8\" (UniqueName: \"kubernetes.io/projected/8fe3690a-68d4-4e16-9871-c258935edb0e-kube-api-access-5r4t8\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:09.054228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054010    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e49f3bf5-4164-4228-bdb2-5145951d33e0-ca-trust-extracted\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.054228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054043    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpq2\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-kube-api-access-9tpq2\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.054228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054071    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-trusted-ca\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.054228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054098    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-installation-pull-secrets\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.054228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054122    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8fe3690a-68d4-4e16-9871-c258935edb0e-config\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:09.054228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054146    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8fe3690a-68d4-4e16-9871-c258935edb0e-trusted-ca\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:09.054228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054192    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.054228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054221    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-bound-sa-token\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.054898 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054249    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-stats-auth\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.054898 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054298    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-image-registry-private-configuration\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.054898 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054387    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e49f3bf5-4164-4228-bdb2-5145951d33e0-ca-trust-extracted\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.054898 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.054485    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-certificates\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.054898 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.054515    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 20:29:09.054898 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.054531    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75757fc68b-5gs8h: secret "image-registry-tls" not found
Jul 30 20:29:09.054898 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.054623    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls podName:e49f3bf5-4164-4228-bdb2-5145951d33e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:09.554601707 +0000 UTC m=+228.015560513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls") pod "image-registry-75757fc68b-5gs8h" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0") : secret "image-registry-tls" not found
Jul 30 20:29:09.055250 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.055191    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-trusted-ca\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.055391 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.055365    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8fe3690a-68d4-4e16-9871-c258935edb0e-config\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:09.055577 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.055552    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8fe3690a-68d4-4e16-9871-c258935edb0e-trusted-ca\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:09.056823 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.056803    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-image-registry-private-configuration\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.057045 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.057018    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fe3690a-68d4-4e16-9871-c258935edb0e-serving-cert\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:09.057122 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.057101    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-installation-pull-secrets\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.065593 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.065566    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-bound-sa-token\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.065675 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.065663    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tpq2\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-kube-api-access-9tpq2\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.066033 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.066011    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r4t8\" (UniqueName: \"kubernetes.io/projected/8fe3690a-68d4-4e16-9871-c258935edb0e-kube-api-access-5r4t8\") pod \"console-operator-5ccd6b445f-7xmcm\" (UID: \"8fe3690a-68d4-4e16-9871-c258935edb0e\") " pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:09.155449 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.155362    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-stats-auth\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.155449 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.155421    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:09.155449 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.155449    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gjmp\" (UniqueName: \"kubernetes.io/projected/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-kube-api-access-5gjmp\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.155686 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.155466    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hschs\" (UniqueName: \"kubernetes.io/projected/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-kube-api-access-hschs\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:09.155686 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.155488    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.155686 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.155504    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-default-certificate\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.155686 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.155519    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.155686 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.155609    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 20:29:09.155686 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.155617    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 20:29:09.156176 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.155957    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:09.655929033 +0000 UTC m=+228.116887836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : configmap references non-existent config key: service-ca.crt
Jul 30 20:29:09.156176 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.156155    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls podName:e1f4663e-6163-4c74-bdc7-36b9204d3ed2 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:09.655974545 +0000 UTC m=+228.116933341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-rf89m" (UID: "e1f4663e-6163-4c74-bdc7-36b9204d3ed2") : secret "samples-operator-tls" not found
Jul 30 20:29:09.156406 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.156386    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:09.656359186 +0000 UTC m=+228.117317992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : secret "router-metrics-certs-default" not found
Jul 30 20:29:09.159045 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.159018    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-stats-auth\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.161320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.161297    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-default-certificate\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.164287 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.164258    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gjmp\" (UniqueName: \"kubernetes.io/projected/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-kube-api-access-5gjmp\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.164756 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.164735    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hschs\" (UniqueName: \"kubernetes.io/projected/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-kube-api-access-hschs\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:09.187356 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.187322    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:09.307964 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.307930    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-7xmcm"]
Jul 30 20:29:09.312567 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:29:09.312542    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe3690a_68d4_4e16_9871_c258935edb0e.slice/crio-d0bf5e5f095260d80961686cc9e72b5e0749f373fbb6b4c66ba3355d6794e24f WatchSource:0}: Error finding container d0bf5e5f095260d80961686cc9e72b5e0749f373fbb6b4c66ba3355d6794e24f: Status 404 returned error can't find the container with id d0bf5e5f095260d80961686cc9e72b5e0749f373fbb6b4c66ba3355d6794e24f
Jul 30 20:29:09.558781 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.558679    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:09.558924 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.558831    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 20:29:09.558924 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.558851    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75757fc68b-5gs8h: secret "image-registry-tls" not found
Jul 30 20:29:09.558924 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.558918    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls podName:e49f3bf5-4164-4228-bdb2-5145951d33e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:10.558902311 +0000 UTC m=+229.019861100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls") pod "image-registry-75757fc68b-5gs8h" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0") : secret "image-registry-tls" not found
Jul 30 20:29:09.659300 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.659271    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.659300 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.659306    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:09.659508 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.659360    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:09.659508 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.659416    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 20:29:09.659508 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.659449    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 20:29:09.659508 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.659501    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:10.659485581 +0000 UTC m=+229.120444377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : configmap references non-existent config key: service-ca.crt
Jul 30 20:29:09.659651 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.659527    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls podName:e1f4663e-6163-4c74-bdc7-36b9204d3ed2 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:10.659520519 +0000 UTC m=+229.120479312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-rf89m" (UID: "e1f4663e-6163-4c74-bdc7-36b9204d3ed2") : secret "samples-operator-tls" not found
Jul 30 20:29:09.659651 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:09.659543    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:10.659536916 +0000 UTC m=+229.120495708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : secret "router-metrics-certs-default" not found
Jul 30 20:29:09.852002 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:09.851961    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" event={"ID":"8fe3690a-68d4-4e16-9871-c258935edb0e","Type":"ContainerStarted","Data":"d0bf5e5f095260d80961686cc9e72b5e0749f373fbb6b4c66ba3355d6794e24f"}
Jul 30 20:29:10.567303 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:10.567265    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:10.567666 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:10.567378    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 20:29:10.567666 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:10.567401    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75757fc68b-5gs8h: secret "image-registry-tls" not found
Jul 30 20:29:10.567666 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:10.567455    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls podName:e49f3bf5-4164-4228-bdb2-5145951d33e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:12.567439863 +0000 UTC m=+231.028398653 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls") pod "image-registry-75757fc68b-5gs8h" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0") : secret "image-registry-tls" not found
Jul 30 20:29:10.668641 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:10.668604    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:10.668641 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:10.668650    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:10.668897 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:10.668764    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:10.668897 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:10.668796    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 20:29:10.668897 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:10.668877    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:12.668856238 +0000 UTC m=+231.129815045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : secret "router-metrics-certs-default" not found
Jul 30 20:29:10.668897 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:10.668878    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 20:29:10.668897 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:10.668897    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:12.66888702 +0000 UTC m=+231.129845817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : configmap references non-existent config key: service-ca.crt
Jul 30 20:29:10.669170 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:10.668929    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls podName:e1f4663e-6163-4c74-bdc7-36b9204d3ed2 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:12.668913426 +0000 UTC m=+231.129872219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-rf89m" (UID: "e1f4663e-6163-4c74-bdc7-36b9204d3ed2") : secret "samples-operator-tls" not found
Jul 30 20:29:11.859666 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:11.859636    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/0.log"
Jul 30 20:29:11.860058 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:11.859676    2547 generic.go:356] "Generic (PLEG): container finished" podID="8fe3690a-68d4-4e16-9871-c258935edb0e" containerID="9649da67e88a5d3282c01894339cc86291c7c80983ee328556f34e8ac23f7991" exitCode=255
Jul 30 20:29:11.860058 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:11.859759    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" event={"ID":"8fe3690a-68d4-4e16-9871-c258935edb0e","Type":"ContainerDied","Data":"9649da67e88a5d3282c01894339cc86291c7c80983ee328556f34e8ac23f7991"}
Jul 30 20:29:11.860058 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:11.859998    2547 scope.go:117] "RemoveContainer" containerID="9649da67e88a5d3282c01894339cc86291c7c80983ee328556f34e8ac23f7991"
Jul 30 20:29:12.584115 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.584079    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:12.584286 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:12.584222    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 20:29:12.584286 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:12.584240    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75757fc68b-5gs8h: secret "image-registry-tls" not found
Jul 30 20:29:12.584356 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:12.584296    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls podName:e49f3bf5-4164-4228-bdb2-5145951d33e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:16.58427873 +0000 UTC m=+235.045237520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls") pod "image-registry-75757fc68b-5gs8h" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0") : secret "image-registry-tls" not found
Jul 30 20:29:12.685033 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.685002    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:12.685205 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.685045    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:12.685205 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.685063    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:12.685205 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:12.685151    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 20:29:12.685205 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:12.685163    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:16.685148536 +0000 UTC m=+235.146107326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : configmap references non-existent config key: service-ca.crt
Jul 30 20:29:12.685205 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:12.685165    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 20:29:12.685205 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:12.685199    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls podName:e1f4663e-6163-4c74-bdc7-36b9204d3ed2 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:16.685187589 +0000 UTC m=+235.146146383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-rf89m" (UID: "e1f4663e-6163-4c74-bdc7-36b9204d3ed2") : secret "samples-operator-tls" not found
Jul 30 20:29:12.685444 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:12.685219    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:16.685205606 +0000 UTC m=+235.146164398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : secret "router-metrics-certs-default" not found
Jul 30 20:29:12.863902 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.863833    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/1.log"
Jul 30 20:29:12.864260 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.864230    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/0.log"
Jul 30 20:29:12.864305 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.864270    2547 generic.go:356] "Generic (PLEG): container finished" podID="8fe3690a-68d4-4e16-9871-c258935edb0e" containerID="8641fe8e65a472b5cec19ba5bfc78b47ea9b30aae783797396ad7f16fd116ce6" exitCode=255
Jul 30 20:29:12.864370 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.864348    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" event={"ID":"8fe3690a-68d4-4e16-9871-c258935edb0e","Type":"ContainerDied","Data":"8641fe8e65a472b5cec19ba5bfc78b47ea9b30aae783797396ad7f16fd116ce6"}
Jul 30 20:29:12.864428 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.864415    2547 scope.go:117] "RemoveContainer" containerID="9649da67e88a5d3282c01894339cc86291c7c80983ee328556f34e8ac23f7991"
Jul 30 20:29:12.864650 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:12.864629    2547 scope.go:117] "RemoveContainer" containerID="8641fe8e65a472b5cec19ba5bfc78b47ea9b30aae783797396ad7f16fd116ce6"
Jul 30 20:29:12.864864 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:12.864843    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-7xmcm_openshift-console-operator(8fe3690a-68d4-4e16-9871-c258935edb0e)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" podUID="8fe3690a-68d4-4e16-9871-c258935edb0e"
Jul 30 20:29:13.443838 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.443795    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m2hjb"]
Jul 30 20:29:13.446449 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.446429    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.448935 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.448916    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 20:29:13.449028 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.448945    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h92tr\""
Jul 30 20:29:13.450258 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.450234    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 20:29:13.450356 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.450299    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 20:29:13.450356 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.450303    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 20:29:13.457695 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.457677    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m2hjb"]
Jul 30 20:29:13.490759 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.490733    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ce2fcd5-97a7-4799-9dcb-5806750659e5-data-volume\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.490879 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.490765    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.490879 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.490784    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cb5k\" (UniqueName: \"kubernetes.io/projected/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-api-access-4cb5k\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.490879 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.490849    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.490991 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.490890    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4ce2fcd5-97a7-4799-9dcb-5806750659e5-crio-socket\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.591269 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.591231    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ce2fcd5-97a7-4799-9dcb-5806750659e5-data-volume\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.591269 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.591271    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.591517 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.591291    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cb5k\" (UniqueName: \"kubernetes.io/projected/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-api-access-4cb5k\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.591517 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:13.591384    2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:13.591517 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.591441    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.591517 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:13.591446    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:14.091430919 +0000 UTC m=+232.552389709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:13.591517 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.591485    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4ce2fcd5-97a7-4799-9dcb-5806750659e5-crio-socket\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.591789 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:13.591557    2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 20:29:13.591789 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:13.591606    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:14.091591945 +0000 UTC m=+232.552550751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : configmap "kube-rbac-proxy" not found
Jul 30 20:29:13.591789 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.591620    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4ce2fcd5-97a7-4799-9dcb-5806750659e5-crio-socket\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.591789 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.591626    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ce2fcd5-97a7-4799-9dcb-5806750659e5-data-volume\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.599455 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.599429    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cb5k\" (UniqueName: \"kubernetes.io/projected/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-api-access-4cb5k\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:13.868255 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.868229    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/1.log"
Jul 30 20:29:13.868621 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:13.868530    2547 scope.go:117] "RemoveContainer" containerID="8641fe8e65a472b5cec19ba5bfc78b47ea9b30aae783797396ad7f16fd116ce6"
Jul 30 20:29:13.868694 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:13.868673    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-7xmcm_openshift-console-operator(8fe3690a-68d4-4e16-9871-c258935edb0e)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" podUID="8fe3690a-68d4-4e16-9871-c258935edb0e"
Jul 30 20:29:14.095546 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:14.095512    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:14.095753 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:14.095554    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:14.095753 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:14.095628    2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 20:29:14.095753 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:14.095670    2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:14.095753 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:14.095677    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:15.095663665 +0000 UTC m=+233.556622458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : configmap "kube-rbac-proxy" not found
Jul 30 20:29:14.095753 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:14.095756    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:15.095737983 +0000 UTC m=+233.556696772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:15.103589 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:15.103559    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:15.104033 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:15.103603    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:15.104033 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:15.103715    2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 20:29:15.104033 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:15.103730    2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:15.104033 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:15.103769    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:17.103754998 +0000 UTC m=+235.564713790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : configmap "kube-rbac-proxy" not found
Jul 30 20:29:15.104033 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:15.103791    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:17.103774371 +0000 UTC m=+235.564733164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:15.418664 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:15.418589    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8x4qs_029812bb-2174-49d1-9bc2-9d576b3c2c80/dns-node-resolver/0.log"
Jul 30 20:29:16.219619 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:16.219591    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8kxwd_17492ecc-9882-4680-9df3-5ef550df1e0d/node-ca/0.log"
Jul 30 20:29:16.614089 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:16.614060    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:16.614253 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:16.614207    2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 20:29:16.614253 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:16.614228    2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75757fc68b-5gs8h: secret "image-registry-tls" not found
Jul 30 20:29:16.614324 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:16.614280    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls podName:e49f3bf5-4164-4228-bdb2-5145951d33e0 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:24.614265045 +0000 UTC m=+243.075223834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls") pod "image-registry-75757fc68b-5gs8h" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0") : secret "image-registry-tls" not found
Jul 30 20:29:16.715190 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:16.715159    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:16.715332 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:16.715208    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:16.715332 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:16.715227    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:16.715332 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:16.715317    2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 20:29:16.715332 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:16.715319    2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 20:29:16.715502 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:16.715362    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:24.715350032 +0000 UTC m=+243.176308826 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : configmap references non-existent config key: service-ca.crt
Jul 30 20:29:16.715502 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:16.715375    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls podName:e1f4663e-6163-4c74-bdc7-36b9204d3ed2 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:24.715369581 +0000 UTC m=+243.176328373 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-rf89m" (UID: "e1f4663e-6163-4c74-bdc7-36b9204d3ed2") : secret "samples-operator-tls" not found
Jul 30 20:29:16.715502 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:16.715387    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:24.715379645 +0000 UTC m=+243.176338435 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : secret "router-metrics-certs-default" not found
Jul 30 20:29:17.019659 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:17.019587    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-48ck8_ae70ec57-f212-4922-8ea3-c2704e36af32/serve-healthcheck-canary/0.log"
Jul 30 20:29:17.118234 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:17.118196    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:17.118361 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:17.118248    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:17.118361 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:17.118310    2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 20:29:17.118361 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:17.118331    2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:17.118361 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:17.118360    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:21.118347419 +0000 UTC m=+239.579306213 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : configmap "kube-rbac-proxy" not found
Jul 30 20:29:17.118486 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:17.118378    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:21.118366202 +0000 UTC m=+239.579324992 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:19.187934 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:19.187840    2547 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:19.187934 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:19.187904    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:19.188587 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:19.188563    2547 scope.go:117] "RemoveContainer" containerID="8641fe8e65a472b5cec19ba5bfc78b47ea9b30aae783797396ad7f16fd116ce6"
Jul 30 20:29:19.189043 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:19.189016    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-7xmcm_openshift-console-operator(8fe3690a-68d4-4e16-9871-c258935edb0e)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" podUID="8fe3690a-68d4-4e16-9871-c258935edb0e"
Jul 30 20:29:21.148068 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:21.148031    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:21.148464 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:21.148091    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:21.148464 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:21.148189    2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:21.148464 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:21.148222    2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 20:29:21.148464 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:21.148265    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:29.148247699 +0000 UTC m=+247.609206496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : secret "insights-runtime-extractor-tls" not found
Jul 30 20:29:21.148464 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:21.148285    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm podName:4ce2fcd5-97a7-4799-9dcb-5806750659e5 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:29.148271962 +0000 UTC m=+247.609230752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2hjb" (UID: "4ce2fcd5-97a7-4799-9dcb-5806750659e5") : configmap "kube-rbac-proxy" not found
Jul 30 20:29:24.677544 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.677511    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:24.680075 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.680049    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") pod \"image-registry-75757fc68b-5gs8h\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") " pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:24.778080 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.778041    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:24.778080 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.778089    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:24.778254 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.778177    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:24.778311 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:24.778293    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle podName:b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:40.778273872 +0000 UTC m=+259.239232662 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle") pod "router-default-68cfb99b6d-9cgw6" (UID: "b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3") : configmap references non-existent config key: service-ca.crt
Jul 30 20:29:24.780619 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.780589    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1f4663e-6163-4c74-bdc7-36b9204d3ed2-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-rf89m\" (UID: \"e1f4663e-6163-4c74-bdc7-36b9204d3ed2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:24.780619 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.780598    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-metrics-certs\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:24.796157 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.796140    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mzftn\""
Jul 30 20:29:24.804226 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.804213    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:24.886317 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.886283    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ftt9p\""
Jul 30 20:29:24.894599 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.894579    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"
Jul 30 20:29:24.934144 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:24.933562    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75757fc68b-5gs8h"]
Jul 30 20:29:24.937069 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:29:24.937035    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49f3bf5_4164_4228_bdb2_5145951d33e0.slice/crio-d0ecb644f866ffe6e6e38d72b1ee687ba92ccee997958c26e734fe436f6c15b2 WatchSource:0}: Error finding container d0ecb644f866ffe6e6e38d72b1ee687ba92ccee997958c26e734fe436f6c15b2: Status 404 returned error can't find the container with id d0ecb644f866ffe6e6e38d72b1ee687ba92ccee997958c26e734fe436f6c15b2
Jul 30 20:29:25.024233 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:25.024196    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m"]
Jul 30 20:29:25.906109 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:25.906063    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m" event={"ID":"e1f4663e-6163-4c74-bdc7-36b9204d3ed2","Type":"ContainerStarted","Data":"f77f11f423ec2f38f53ef5befce3500e0d4c9a7a12bb75b7a9eba274d660726d"}
Jul 30 20:29:25.907784 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:25.907752    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h" event={"ID":"e49f3bf5-4164-4228-bdb2-5145951d33e0","Type":"ContainerStarted","Data":"87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3"}
Jul 30 20:29:25.907940 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:25.907808    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h" event={"ID":"e49f3bf5-4164-4228-bdb2-5145951d33e0","Type":"ContainerStarted","Data":"d0ecb644f866ffe6e6e38d72b1ee687ba92ccee997958c26e734fe436f6c15b2"}
Jul 30 20:29:25.907940 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:25.907849    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:25.929557 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:25.929511    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h" podStartSLOduration=17.92949274 podStartE2EDuration="17.92949274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:29:25.9280086 +0000 UTC m=+244.388967413" watchObservedRunningTime="2026-07-30 20:29:25.92949274 +0000 UTC m=+244.390451556"
Jul 30 20:29:27.914756 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:27.914717    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m" event={"ID":"e1f4663e-6163-4c74-bdc7-36b9204d3ed2","Type":"ContainerStarted","Data":"e1ac1468efc3463c6fabb4df334e4ccbeb67571adc9b28d6bfe3c136ed9d80c4"}
Jul 30 20:29:27.915172 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:27.914779    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m" event={"ID":"e1f4663e-6163-4c74-bdc7-36b9204d3ed2","Type":"ContainerStarted","Data":"e705c6ac72b06aad7af82e17e37e51809289ac00a30b87080744ab5979f103de"}
Jul 30 20:29:27.931936 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:27.931885    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-rf89m" podStartSLOduration=18.135019532 podStartE2EDuration="19.931868724s" totalImagesPullingTime="1.796849192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:29:27.93038697 +0000 UTC m=+246.391345781" watchObservedRunningTime="2026-07-30 20:29:27.931868724 +0000 UTC m=+246.392827536"
Jul 30 20:29:29.218088 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:29.218040    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:29.218088 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:29.218091    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:29.218729 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:29.218678    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ce2fcd5-97a7-4799-9dcb-5806750659e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:29.220602 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:29.220577    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ce2fcd5-97a7-4799-9dcb-5806750659e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2hjb\" (UID: \"4ce2fcd5-97a7-4799-9dcb-5806750659e5\") " pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:29.358633 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:29.358595    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h92tr\""
Jul 30 20:29:29.367067 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:29.367037    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m2hjb"
Jul 30 20:29:29.499622 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:29.499543    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m2hjb"]
Jul 30 20:29:29.504014 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:29:29.503988    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce2fcd5_97a7_4799_9dcb_5806750659e5.slice/crio-4088839cec9761f00c4f6eac8e4807bb10925cb56b1bfd6315190f35dd585fc5 WatchSource:0}: Error finding container 4088839cec9761f00c4f6eac8e4807bb10925cb56b1bfd6315190f35dd585fc5: Status 404 returned error can't find the container with id 4088839cec9761f00c4f6eac8e4807bb10925cb56b1bfd6315190f35dd585fc5
Jul 30 20:29:29.922352 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:29.922318    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2hjb" event={"ID":"4ce2fcd5-97a7-4799-9dcb-5806750659e5","Type":"ContainerStarted","Data":"2bf3ec617636983b3c3ca979989a34386c1ff00945da00c708d6b66a93b35de2"}
Jul 30 20:29:29.922534 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:29.922372    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2hjb" event={"ID":"4ce2fcd5-97a7-4799-9dcb-5806750659e5","Type":"ContainerStarted","Data":"4088839cec9761f00c4f6eac8e4807bb10925cb56b1bfd6315190f35dd585fc5"}
Jul 30 20:29:30.927185 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:30.927145    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2hjb" event={"ID":"4ce2fcd5-97a7-4799-9dcb-5806750659e5","Type":"ContainerStarted","Data":"76ae0bf6d28c474dcd2faa71eb72ad5a126b751c0a61c3f31d27fefd2c2848e3"}
Jul 30 20:29:32.849188 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:32.849146    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:29:32.851715 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:32.851678    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5679c74-1e69-4be8-90d8-3bf6aa168ee4-metrics-certs\") pod \"network-metrics-daemon-l6rqg\" (UID: \"d5679c74-1e69-4be8-90d8-3bf6aa168ee4\") " pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:29:32.935947 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:32.935911    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2hjb" event={"ID":"4ce2fcd5-97a7-4799-9dcb-5806750659e5","Type":"ContainerStarted","Data":"841a489fadd7cf889c4438871f2037c386fc73ef807e57f914290fc1c22df195"}
Jul 30 20:29:32.955458 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:32.955398    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m2hjb" podStartSLOduration=17.61796423 podStartE2EDuration="19.955380988s" totalImagesPullingTime="2.337416758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:29:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:29:32.954419714 +0000 UTC m=+251.415378526" watchObservedRunningTime="2026-07-30 20:29:32.955380988 +0000 UTC m=+251.416339800"
Jul 30 20:29:33.070336 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:33.070299    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4nz9n\""
Jul 30 20:29:33.077802 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:33.077761    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l6rqg"
Jul 30 20:29:33.215234 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:33.215191    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l6rqg"]
Jul 30 20:29:33.218082 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:29:33.218053    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5679c74_1e69_4be8_90d8_3bf6aa168ee4.slice/crio-efc442d0ebed93c9b28e0ef820409de4d36c502c63f0363b81ad1d9f4f919394 WatchSource:0}: Error finding container efc442d0ebed93c9b28e0ef820409de4d36c502c63f0363b81ad1d9f4f919394: Status 404 returned error can't find the container with id efc442d0ebed93c9b28e0ef820409de4d36c502c63f0363b81ad1d9f4f919394
Jul 30 20:29:33.942511 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:33.942460    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l6rqg" event={"ID":"d5679c74-1e69-4be8-90d8-3bf6aa168ee4","Type":"ContainerStarted","Data":"efc442d0ebed93c9b28e0ef820409de4d36c502c63f0363b81ad1d9f4f919394"}
Jul 30 20:29:34.947299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:34.947261    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l6rqg" event={"ID":"d5679c74-1e69-4be8-90d8-3bf6aa168ee4","Type":"ContainerStarted","Data":"21b67203b45674ea3b852fb717d7039a3d4d88ae8ba3a005062f0240b1064458"}
Jul 30 20:29:34.947736 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:34.947348    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l6rqg" event={"ID":"d5679c74-1e69-4be8-90d8-3bf6aa168ee4","Type":"ContainerStarted","Data":"a12949ebb4317cd9a6ce7498a1938544a7752cc6d2d452608ffc85a50a53a0f4"}
Jul 30 20:29:34.963781 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:34.963725    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l6rqg" podStartSLOduration=252.113013178 podStartE2EDuration="4m12.963678679s" totalImagesPullingTime="850.665501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:29:34.963287434 +0000 UTC m=+253.424246310" watchObservedRunningTime="2026-07-30 20:29:34.963678679 +0000 UTC m=+253.424637491"
Jul 30 20:29:35.067150 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:35.067122    2547 scope.go:117] "RemoveContainer" containerID="8641fe8e65a472b5cec19ba5bfc78b47ea9b30aae783797396ad7f16fd116ce6"
Jul 30 20:29:35.952187 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:35.952153    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:29:35.952594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:35.952542    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/1.log"
Jul 30 20:29:35.952594 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:35.952575    2547 generic.go:356] "Generic (PLEG): container finished" podID="8fe3690a-68d4-4e16-9871-c258935edb0e" containerID="6ab8e70d99d081b47a7dcdbae743912a2bd15f71b95d98309da887ba3c0272b8" exitCode=255
Jul 30 20:29:35.952689 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:35.952659    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" event={"ID":"8fe3690a-68d4-4e16-9871-c258935edb0e","Type":"ContainerDied","Data":"6ab8e70d99d081b47a7dcdbae743912a2bd15f71b95d98309da887ba3c0272b8"}
Jul 30 20:29:35.952764 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:35.952750    2547 scope.go:117] "RemoveContainer" containerID="8641fe8e65a472b5cec19ba5bfc78b47ea9b30aae783797396ad7f16fd116ce6"
Jul 30 20:29:35.953129 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:35.953102    2547 scope.go:117] "RemoveContainer" containerID="6ab8e70d99d081b47a7dcdbae743912a2bd15f71b95d98309da887ba3c0272b8"
Jul 30 20:29:35.953275 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:35.953243    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-7xmcm_openshift-console-operator(8fe3690a-68d4-4e16-9871-c258935edb0e)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" podUID="8fe3690a-68d4-4e16-9871-c258935edb0e"
Jul 30 20:29:36.957786 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:36.957758    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:29:38.358690 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:38.358641    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75757fc68b-5gs8h"]
Jul 30 20:29:39.187985 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:39.187945    2547 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:39.188205 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:39.188005    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:29:39.188324 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:39.188311    2547 scope.go:117] "RemoveContainer" containerID="6ab8e70d99d081b47a7dcdbae743912a2bd15f71b95d98309da887ba3c0272b8"
Jul 30 20:29:39.188483 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:39.188463    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-7xmcm_openshift-console-operator(8fe3690a-68d4-4e16-9871-c258935edb0e)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" podUID="8fe3690a-68d4-4e16-9871-c258935edb0e"
Jul 30 20:29:40.807192 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:40.807154    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:40.807821 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:40.807793    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3-service-ca-bundle\") pod \"router-default-68cfb99b6d-9cgw6\" (UID: \"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3\") " pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:41.091442 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:41.091413    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2nwg9\""
Jul 30 20:29:41.099612 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:41.099595    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:41.244166 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:41.244130    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68cfb99b6d-9cgw6"]
Jul 30 20:29:41.977483 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:41.977431    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68cfb99b6d-9cgw6" event={"ID":"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3","Type":"ContainerStarted","Data":"d951375817c0b0d1534df62db0e281064af16ff33eaec4226e78e6ded86e30c9"}
Jul 30 20:29:41.978002 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:41.977505    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68cfb99b6d-9cgw6" event={"ID":"b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3","Type":"ContainerStarted","Data":"b82397a428fa034fcb1342fad21f60fd56631fa12f502901505288ce67392e7e"}
Jul 30 20:29:42.001319 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:42.001259    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68cfb99b6d-9cgw6" podStartSLOduration=34.00124077 podStartE2EDuration="34.00124077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:29:41.999171121 +0000 UTC m=+260.460129934" watchObservedRunningTime="2026-07-30 20:29:42.00124077 +0000 UTC m=+260.462199581"
Jul 30 20:29:42.100345 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:42.100313    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:42.102959 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:42.102936    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:42.980656 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:42.980582    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:42.981899 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:42.981879    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68cfb99b6d-9cgw6"
Jul 30 20:29:48.364306 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:48.364270    2547 patch_prober.go:28] interesting pod/image-registry-75757fc68b-5gs8h container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 30 20:29:48.364695 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:48.364336    2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h" podUID="e49f3bf5-4164-4228-bdb2-5145951d33e0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 20:29:50.066643 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:50.066609    2547 scope.go:117] "RemoveContainer" containerID="6ab8e70d99d081b47a7dcdbae743912a2bd15f71b95d98309da887ba3c0272b8"
Jul 30 20:29:50.067105 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:50.066831    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-7xmcm_openshift-console-operator(8fe3690a-68d4-4e16-9871-c258935edb0e)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" podUID="8fe3690a-68d4-4e16-9871-c258935edb0e"
Jul 30 20:29:51.650078 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.650045    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"]
Jul 30 20:29:51.653497 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.653478    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.656136 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.656116    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 30 20:29:51.656211 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.656116    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ljhd5\""
Jul 30 20:29:51.657444 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.657427    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 20:29:51.657545 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.657455    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 30 20:29:51.657545 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.657450    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 20:29:51.657545 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.657490    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 20:29:51.657688 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.657450    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 30 20:29:51.664809 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.664785    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6x9d6"]
Jul 30 20:29:51.667885 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.667859    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"]
Jul 30 20:29:51.667972 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.667951    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.670236 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.670214    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 20:29:51.670346 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.670247    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 20:29:51.670346 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.670225    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nwh68\""
Jul 30 20:29:51.670346 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.670313    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 20:29:51.798495 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798461    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.798495 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798494    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-tls\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.798731 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798513    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.798731 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798533    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-accelerators-collector-config\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.798731 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798611    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-textfile\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.798731 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798648    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4b35782-fbda-4b14-b357-aeef3e1a6265-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.798731 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798676    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.798731 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798723    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6bc63d6-58e6-481c-8900-bd67d7942342-root\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.798972 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798751    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc63d6-58e6-481c-8900-bd67d7942342-metrics-client-ca\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.798972 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798766    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czr5w\" (UniqueName: \"kubernetes.io/projected/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-api-access-czr5w\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.798972 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798801    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-wtmp\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.798972 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798846    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f4b35782-fbda-4b14-b357-aeef3e1a6265-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.798972 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798883    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvl4r\" (UniqueName: \"kubernetes.io/projected/e6bc63d6-58e6-481c-8900-bd67d7942342-kube-api-access-nvl4r\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.798972 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798906    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6bc63d6-58e6-481c-8900-bd67d7942342-sys\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.798972 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.798939    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.899792 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.899751    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.899965 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.899800    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.899965 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.899835    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-tls\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.899965 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.899860    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.899965 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.899895    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-accelerators-collector-config\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.899965 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.899923    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-textfile\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.899965 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.899945    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4b35782-fbda-4b14-b357-aeef3e1a6265-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.900299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.899974    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.900299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900007    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6bc63d6-58e6-481c-8900-bd67d7942342-root\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.900299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900035    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc63d6-58e6-481c-8900-bd67d7942342-metrics-client-ca\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.900299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900060    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czr5w\" (UniqueName: \"kubernetes.io/projected/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-api-access-czr5w\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.900299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900104    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-wtmp\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.900299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900183    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6bc63d6-58e6-481c-8900-bd67d7942342-root\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.900598 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900542    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f4b35782-fbda-4b14-b357-aeef3e1a6265-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.900660 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900592    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-textfile\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.900660 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900614    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl4r\" (UniqueName: \"kubernetes.io/projected/e6bc63d6-58e6-481c-8900-bd67d7942342-kube-api-access-nvl4r\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.900660 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900643    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6bc63d6-58e6-481c-8900-bd67d7942342-sys\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.900839 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900752    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6bc63d6-58e6-481c-8900-bd67d7942342-sys\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.900895 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900866    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4b35782-fbda-4b14-b357-aeef3e1a6265-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.900952 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.900914    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f4b35782-fbda-4b14-b357-aeef3e1a6265-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.901034 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.901013    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-wtmp\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.901472 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.901436    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc63d6-58e6-481c-8900-bd67d7942342-metrics-client-ca\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.901600 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.901506    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.901600 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.901562    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-accelerators-collector-config\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.902511 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.902492    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-tls\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.902719 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.902680    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6bc63d6-58e6-481c-8900-bd67d7942342-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.903001 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.902983    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.903065 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.903045    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.908084 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.908063    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvl4r\" (UniqueName: \"kubernetes.io/projected/e6bc63d6-58e6-481c-8900-bd67d7942342-kube-api-access-nvl4r\") pod \"node-exporter-6x9d6\" (UID: \"e6bc63d6-58e6-481c-8900-bd67d7942342\") " pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.908397 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.908379    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czr5w\" (UniqueName: \"kubernetes.io/projected/f4b35782-fbda-4b14-b357-aeef3e1a6265-kube-api-access-czr5w\") pod \"kube-state-metrics-6bd9664c4-6mp2w\" (UID: \"f4b35782-fbda-4b14-b357-aeef3e1a6265\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.962403 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.962373    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"
Jul 30 20:29:51.976609 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:51.976585    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6x9d6"
Jul 30 20:29:51.987478 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:29:51.987445    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bc63d6_58e6_481c_8900_bd67d7942342.slice/crio-53dee1e1652c8edccf16a966d2a9e07574822d861c1147ab0c3a23db3ae7acd8 WatchSource:0}: Error finding container 53dee1e1652c8edccf16a966d2a9e07574822d861c1147ab0c3a23db3ae7acd8: Status 404 returned error can't find the container with id 53dee1e1652c8edccf16a966d2a9e07574822d861c1147ab0c3a23db3ae7acd8
Jul 30 20:29:52.018099 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.017386    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6x9d6" event={"ID":"e6bc63d6-58e6-481c-8900-bd67d7942342","Type":"ContainerStarted","Data":"53dee1e1652c8edccf16a966d2a9e07574822d861c1147ab0c3a23db3ae7acd8"}
Jul 30 20:29:52.104078 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.104053    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w"]
Jul 30 20:29:52.106499 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:29:52.106472    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b35782_fbda_4b14_b357_aeef3e1a6265.slice/crio-6ab7ff8f332521f3fceaa2a3e8241f1b7a9f2e140b8135a0c29ee6dc37ec25b8 WatchSource:0}: Error finding container 6ab7ff8f332521f3fceaa2a3e8241f1b7a9f2e140b8135a0c29ee6dc37ec25b8: Status 404 returned error can't find the container with id 6ab7ff8f332521f3fceaa2a3e8241f1b7a9f2e140b8135a0c29ee6dc37ec25b8
Jul 30 20:29:52.776138 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.776104    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 20:29:52.780360 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.780338    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.784994 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.784958    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 30 20:29:52.784994 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.784973    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 30 20:29:52.785191 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.784958    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 30 20:29:52.785191 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.784974    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 30 20:29:52.785191 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.785168    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 30 20:29:52.785653 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.785636    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 30 20:29:52.785813 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.785736    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 30 20:29:52.785995 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.785969    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 30 20:29:52.786506 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.786487    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vtgvg\""
Jul 30 20:29:52.791015 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.790988    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 30 20:29:52.794864 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.794846    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 20:29:52.809108 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809081    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809229 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809125    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-web-config\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809229 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809158    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809229 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809188    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809229 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809221    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809440 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809247    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809440 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809286    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5kxz\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-kube-api-access-x5kxz\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809440 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809362    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-volume\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809440 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809386    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809440 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809415    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809440 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809438    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809729 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809520    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-out\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.809729 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.809585    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.910403 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910368    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-out\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.910546 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910433    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.910546 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910508    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.910655 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910540    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-web-config\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.910729 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910642    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.910729 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910717    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.910833 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910765    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.910833 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910791    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.910833 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:52.910796    2547 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 30 20:29:52.910833 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910814    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5kxz\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-kube-api-access-x5kxz\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.911039 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:29:52.910864    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-main-tls podName:de08378f-1a11-480b-83c7-71ea2ce9ece2 nodeName:}" failed. No retries permitted until 2026-07-30 20:29:53.410842496 +0000 UTC m=+271.871801306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2") : secret "alertmanager-main-tls" not found
Jul 30 20:29:52.911039 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910918    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-volume\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.911039 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910951    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.911039 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.910991    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.911039 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.911013    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.911504 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.911379    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.912005 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.911902    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.912005 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.911922    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.913876 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.913831    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-web-config\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.913986 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.913918    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.914213 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.914180    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-out\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.914349 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.914328    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.914453 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.914433    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.914963 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.914928    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-volume\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.915229 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.915209    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.915295 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.915235    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:52.921845 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:52.921826    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5kxz\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-kube-api-access-x5kxz\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:53.024112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:53.023993    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6x9d6" event={"ID":"e6bc63d6-58e6-481c-8900-bd67d7942342","Type":"ContainerStarted","Data":"e514c19c48380fd4b6cd52fd11cf4b5cf5086733750f1d16e5d2052d86e47864"}
Jul 30 20:29:53.025752 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:53.025726    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w" event={"ID":"f4b35782-fbda-4b14-b357-aeef3e1a6265","Type":"ContainerStarted","Data":"6ab7ff8f332521f3fceaa2a3e8241f1b7a9f2e140b8135a0c29ee6dc37ec25b8"}
Jul 30 20:29:53.415896 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:53.415786    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:53.418639 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:53.418614    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:53.693860 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:53.693838    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:29:53.851895 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:53.849160    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 20:29:54.031222 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:54.031130    2547 generic.go:356] "Generic (PLEG): container finished" podID="e6bc63d6-58e6-481c-8900-bd67d7942342" containerID="e514c19c48380fd4b6cd52fd11cf4b5cf5086733750f1d16e5d2052d86e47864" exitCode=0
Jul 30 20:29:54.031394 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:54.031228    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6x9d6" event={"ID":"e6bc63d6-58e6-481c-8900-bd67d7942342","Type":"ContainerDied","Data":"e514c19c48380fd4b6cd52fd11cf4b5cf5086733750f1d16e5d2052d86e47864"}
Jul 30 20:29:54.033601 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:54.033577    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w" event={"ID":"f4b35782-fbda-4b14-b357-aeef3e1a6265","Type":"ContainerStarted","Data":"ec133e407cea1b84cbda253584b186c8cd6b61f19c2a06db3f068aaca13b1a35"}
Jul 30 20:29:54.033749 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:54.033635    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w" event={"ID":"f4b35782-fbda-4b14-b357-aeef3e1a6265","Type":"ContainerStarted","Data":"630f71973d00d69257c1fecf920e6ea75fc6528201782f888ed79a0cae7067ca"}
Jul 30 20:29:54.033749 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:54.033666    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w" event={"ID":"f4b35782-fbda-4b14-b357-aeef3e1a6265","Type":"ContainerStarted","Data":"f219c58918ec9699174a2a9d4cd69486d37438f1e510850563d3e6a87cb69466"}
Jul 30 20:29:54.034827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:54.034806    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerStarted","Data":"9fc6bb5813aeb852ba2a09d0ac09bd2b6984871e63ebb3b729d3cc432f400f28"}
Jul 30 20:29:54.072380 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:54.072332    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-6mp2w" podStartSLOduration=1.499345324 podStartE2EDuration="3.07231474s" totalImagesPullingTime="1.572969416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:29:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:29:54.071531224 +0000 UTC m=+272.532490038" watchObservedRunningTime="2026-07-30 20:29:54.07231474 +0000 UTC m=+272.533273556"
Jul 30 20:29:55.040440 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.040401    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6x9d6" event={"ID":"e6bc63d6-58e6-481c-8900-bd67d7942342","Type":"ContainerStarted","Data":"9e4344bf36937f33aad420365086f0b5cab180879d49e5cf71d0919ed4924629"}
Jul 30 20:29:55.040840 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.040471    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6x9d6" event={"ID":"e6bc63d6-58e6-481c-8900-bd67d7942342","Type":"ContainerStarted","Data":"659f85a1d88139acd8c06ba26991e803d43bd7436871ff87c5feacc3181d132a"}
Jul 30 20:29:55.061376 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.061334    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6x9d6" podStartSLOduration=3.039784278 podStartE2EDuration="4.061314734s" totalImagesPullingTime="940.471731ms" totalInitContainerRuntime="81.058725ms" isStatefulPod=true podCreationTimestamp="2026-07-30 20:29:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:29:55.058923573 +0000 UTC m=+273.519882386" watchObservedRunningTime="2026-07-30 20:29:55.061314734 +0000 UTC m=+273.522273548"
Jul 30 20:29:55.944845 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.944809    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"]
Jul 30 20:29:55.947948 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.947930    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:55.950844 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.950823    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 30 20:29:55.952050 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.952034    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-h4x9v\""
Jul 30 20:29:55.952165 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.952146    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3n61j51m2958t\""
Jul 30 20:29:55.952223 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.952203    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 30 20:29:55.952309 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.952292    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 30 20:29:55.952365 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.952298    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 30 20:29:55.957138 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:55.957120    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"]
Jul 30 20:29:56.040861 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.040823    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e1305d-550b-425a-a804-b73dad9fdd54-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.041292 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.040888    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4e1305d-550b-425a-a804-b73dad9fdd54-metrics-server-audit-profiles\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.041292 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.040956    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4e1305d-550b-425a-a804-b73dad9fdd54-client-ca-bundle\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.041292 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.040989    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmxb4\" (UniqueName: \"kubernetes.io/projected/a4e1305d-550b-425a-a804-b73dad9fdd54-kube-api-access-kmxb4\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.041292 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.041045    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4e1305d-550b-425a-a804-b73dad9fdd54-audit-log\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.041292 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.041073    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4e1305d-550b-425a-a804-b73dad9fdd54-secret-metrics-server-client-certs\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.041292 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.041095    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4e1305d-550b-425a-a804-b73dad9fdd54-secret-metrics-server-tls\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.045776 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.045750    2547 generic.go:356] "Generic (PLEG): container finished" podID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerID="bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc" exitCode=0
Jul 30 20:29:56.045883 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.045837    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerDied","Data":"bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc"}
Jul 30 20:29:56.142378 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.142340    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmxb4\" (UniqueName: \"kubernetes.io/projected/a4e1305d-550b-425a-a804-b73dad9fdd54-kube-api-access-kmxb4\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.142574 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.142554    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4e1305d-550b-425a-a804-b73dad9fdd54-audit-log\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.142778 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.142758    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4e1305d-550b-425a-a804-b73dad9fdd54-secret-metrics-server-client-certs\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.142849 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.142794    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4e1305d-550b-425a-a804-b73dad9fdd54-secret-metrics-server-tls\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.142906 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.142856    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e1305d-550b-425a-a804-b73dad9fdd54-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.143210 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.143011    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4e1305d-550b-425a-a804-b73dad9fdd54-audit-log\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.143210 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.143019    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4e1305d-550b-425a-a804-b73dad9fdd54-metrics-server-audit-profiles\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.143210 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.143090    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4e1305d-550b-425a-a804-b73dad9fdd54-client-ca-bundle\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.143667 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.143520    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e1305d-550b-425a-a804-b73dad9fdd54-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.144181 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.144156    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4e1305d-550b-425a-a804-b73dad9fdd54-metrics-server-audit-profiles\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.145561 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.145535    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4e1305d-550b-425a-a804-b73dad9fdd54-secret-metrics-server-client-certs\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.145644 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.145629    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4e1305d-550b-425a-a804-b73dad9fdd54-secret-metrics-server-tls\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.145685 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.145653    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4e1305d-550b-425a-a804-b73dad9fdd54-client-ca-bundle\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.150037 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.150021    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmxb4\" (UniqueName: \"kubernetes.io/projected/a4e1305d-550b-425a-a804-b73dad9fdd54-kube-api-access-kmxb4\") pod \"metrics-server-65f7ff9d9b-bghbb\" (UID: \"a4e1305d-550b-425a-a804-b73dad9fdd54\") " pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.257046 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.256961    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:29:56.386212 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:56.386135    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"]
Jul 30 20:29:56.389157 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:29:56.389130    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e1305d_550b_425a_a804_b73dad9fdd54.slice/crio-407b7096d9b4c9a96e9671035b381005b7b4b9c8b1a9eb59f3e7b15d1219a261 WatchSource:0}: Error finding container 407b7096d9b4c9a96e9671035b381005b7b4b9c8b1a9eb59f3e7b15d1219a261: Status 404 returned error can't find the container with id 407b7096d9b4c9a96e9671035b381005b7b4b9c8b1a9eb59f3e7b15d1219a261
Jul 30 20:29:57.051243 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:57.051206    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb" event={"ID":"a4e1305d-550b-425a-a804-b73dad9fdd54","Type":"ContainerStarted","Data":"407b7096d9b4c9a96e9671035b381005b7b4b9c8b1a9eb59f3e7b15d1219a261"}
Jul 30 20:29:58.364638 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:58.364612    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:29:59.061767 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:59.061732    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerStarted","Data":"e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795"}
Jul 30 20:29:59.061927 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:59.061805    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerStarted","Data":"bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb"}
Jul 30 20:29:59.061927 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:59.061844    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerStarted","Data":"2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d"}
Jul 30 20:29:59.061927 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:59.061870    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerStarted","Data":"e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970"}
Jul 30 20:29:59.061927 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:59.061894    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerStarted","Data":"878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49"}
Jul 30 20:29:59.063224 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:59.063186    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb" event={"ID":"a4e1305d-550b-425a-a804-b73dad9fdd54","Type":"ContainerStarted","Data":"50b598f1a1472335d55e0a1b55bc29105fc65526a4203f69921fb2342a9935ee"}
Jul 30 20:29:59.079472 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:29:59.079430    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb" podStartSLOduration=2.291860416 podStartE2EDuration="4.07941505s" totalImagesPullingTime="1.787554634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:29:59.078846517 +0000 UTC m=+277.539805369" watchObservedRunningTime="2026-07-30 20:29:59.07941505 +0000 UTC m=+277.540373861"
Jul 30 20:30:00.071062 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:00.071023    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerStarted","Data":"59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4"}
Jul 30 20:30:00.099834 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:00.099769    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.8957859089999998 podStartE2EDuration="8.099752506s" totalImagesPullingTime="4.187307432s" totalInitContainerRuntime="16.659165ms" isStatefulPod=false podCreationTimestamp="2026-07-30 20:29:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:30:00.098930229 +0000 UTC m=+278.559889041" watchObservedRunningTime="2026-07-30 20:30:00.099752506 +0000 UTC m=+278.560711319"
Jul 30 20:30:02.068250 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:02.068220    2547 scope.go:117] "RemoveContainer" containerID="6ab8e70d99d081b47a7dcdbae743912a2bd15f71b95d98309da887ba3c0272b8"
Jul 30 20:30:02.797994 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:02.797959    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-gdngc"]
Jul 30 20:30:02.800258 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:02.800236    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-gdngc"
Jul 30 20:30:02.803056 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:02.803027    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 20:30:02.803056 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:02.803043    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rt7hp\""
Jul 30 20:30:02.803056 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:02.803030    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 20:30:02.813173 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:02.813151    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-gdngc"]
Jul 30 20:30:02.901356 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:02.901320    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcbzw\" (UniqueName: \"kubernetes.io/projected/cba71dcb-757a-4991-8831-6899b4e8876a-kube-api-access-wcbzw\") pod \"downloads-77c974ffcf-gdngc\" (UID: \"cba71dcb-757a-4991-8831-6899b4e8876a\") " pod="openshift-console/downloads-77c974ffcf-gdngc"
Jul 30 20:30:03.002657 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.002617    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcbzw\" (UniqueName: \"kubernetes.io/projected/cba71dcb-757a-4991-8831-6899b4e8876a-kube-api-access-wcbzw\") pod \"downloads-77c974ffcf-gdngc\" (UID: \"cba71dcb-757a-4991-8831-6899b4e8876a\") " pod="openshift-console/downloads-77c974ffcf-gdngc"
Jul 30 20:30:03.011338 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.011311    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcbzw\" (UniqueName: \"kubernetes.io/projected/cba71dcb-757a-4991-8831-6899b4e8876a-kube-api-access-wcbzw\") pod \"downloads-77c974ffcf-gdngc\" (UID: \"cba71dcb-757a-4991-8831-6899b4e8876a\") " pod="openshift-console/downloads-77c974ffcf-gdngc"
Jul 30 20:30:03.083661 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.083633    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:30:03.084119 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.083752    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" event={"ID":"8fe3690a-68d4-4e16-9871-c258935edb0e","Type":"ContainerStarted","Data":"e5b9b80ee37bd87f058d006eebecbcf1df42a3003822b173d324c0bab9bd0df8"}
Jul 30 20:30:03.084119 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.084039    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:30:03.088993 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.088975    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm"
Jul 30 20:30:03.102882 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.102841    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-7xmcm" podStartSLOduration=53.340257641 podStartE2EDuration="55.10282819s" totalImagesPullingTime="1.762570549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:30:03.101836339 +0000 UTC m=+281.562795156" watchObservedRunningTime="2026-07-30 20:30:03.10282819 +0000 UTC m=+281.563787002"
Jul 30 20:30:03.109491 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.109471    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-gdngc"
Jul 30 20:30:03.247538 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.247508    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-gdngc"]
Jul 30 20:30:03.379791 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.379666    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h" podUID="e49f3bf5-4164-4228-bdb2-5145951d33e0" containerName="registry" containerID="cri-o://87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3" gracePeriod=30
Jul 30 20:30:03.618832 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.618809    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:30:03.659093 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:30:03.658999    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tjtsk" podUID="4bfaea5d-a4b2-42a0-889d-29b157f75146"
Jul 30 20:30:03.708290 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.708256    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") pod \"e49f3bf5-4164-4228-bdb2-5145951d33e0\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") "
Jul 30 20:30:03.708290 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.708298    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-trusted-ca\") pod \"e49f3bf5-4164-4228-bdb2-5145951d33e0\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") "
Jul 30 20:30:03.708514 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.708331    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-image-registry-private-configuration\") pod \"e49f3bf5-4164-4228-bdb2-5145951d33e0\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") "
Jul 30 20:30:03.708514 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.708365    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-installation-pull-secrets\") pod \"e49f3bf5-4164-4228-bdb2-5145951d33e0\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") "
Jul 30 20:30:03.708514 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.708404    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-bound-sa-token\") pod \"e49f3bf5-4164-4228-bdb2-5145951d33e0\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") "
Jul 30 20:30:03.708514 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.708441    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-certificates\") pod \"e49f3bf5-4164-4228-bdb2-5145951d33e0\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") "
Jul 30 20:30:03.708514 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.708495    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e49f3bf5-4164-4228-bdb2-5145951d33e0-ca-trust-extracted\") pod \"e49f3bf5-4164-4228-bdb2-5145951d33e0\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") "
Jul 30 20:30:03.708793 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.708520    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tpq2\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-kube-api-access-9tpq2\") pod \"e49f3bf5-4164-4228-bdb2-5145951d33e0\" (UID: \"e49f3bf5-4164-4228-bdb2-5145951d33e0\") "
Jul 30 20:30:03.708949 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.708919    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e49f3bf5-4164-4228-bdb2-5145951d33e0" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:30:03.709088 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.709055    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e49f3bf5-4164-4228-bdb2-5145951d33e0" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:30:03.710882 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.710853    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e49f3bf5-4164-4228-bdb2-5145951d33e0" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:30:03.711293 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.711267    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e49f3bf5-4164-4228-bdb2-5145951d33e0" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:30:03.711293 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.711279    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e49f3bf5-4164-4228-bdb2-5145951d33e0" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:30:03.711416 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.711342    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-kube-api-access-9tpq2" (OuterVolumeSpecName: "kube-api-access-9tpq2") pod "e49f3bf5-4164-4228-bdb2-5145951d33e0" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0"). InnerVolumeSpecName "kube-api-access-9tpq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:30:03.711416 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.711355    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e49f3bf5-4164-4228-bdb2-5145951d33e0" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:30:03.716526 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.716503    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49f3bf5-4164-4228-bdb2-5145951d33e0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e49f3bf5-4164-4228-bdb2-5145951d33e0" (UID: "e49f3bf5-4164-4228-bdb2-5145951d33e0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:30:03.810023 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.809978    2547 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-installation-pull-secrets\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:30:03.810023 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.810020    2547 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-bound-sa-token\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:30:03.810239 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.810035    2547 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-certificates\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:30:03.810239 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.810051    2547 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e49f3bf5-4164-4228-bdb2-5145951d33e0-ca-trust-extracted\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:30:03.810239 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.810064    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tpq2\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-kube-api-access-9tpq2\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:30:03.810239 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.810078    2547 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e49f3bf5-4164-4228-bdb2-5145951d33e0-registry-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:30:03.810239 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.810091    2547 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e49f3bf5-4164-4228-bdb2-5145951d33e0-trusted-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:30:03.810239 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:03.810105    2547 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e49f3bf5-4164-4228-bdb2-5145951d33e0-image-registry-private-configuration\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:30:04.088955 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.088912    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-gdngc" event={"ID":"cba71dcb-757a-4991-8831-6899b4e8876a","Type":"ContainerStarted","Data":"45694a7b1e0f7ca244eb0aa16d90d54165a7cb9f6fd8400e19333880d812d055"}
Jul 30 20:30:04.090285 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.090260    2547 generic.go:356] "Generic (PLEG): container finished" podID="e49f3bf5-4164-4228-bdb2-5145951d33e0" containerID="87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3" exitCode=0
Jul 30 20:30:04.090412 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.090321    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h"
Jul 30 20:30:04.090412 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.090343    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h" event={"ID":"e49f3bf5-4164-4228-bdb2-5145951d33e0","Type":"ContainerDied","Data":"87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3"}
Jul 30 20:30:04.090412 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.090398    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75757fc68b-5gs8h" event={"ID":"e49f3bf5-4164-4228-bdb2-5145951d33e0","Type":"ContainerDied","Data":"d0ecb644f866ffe6e6e38d72b1ee687ba92ccee997958c26e734fe436f6c15b2"}
Jul 30 20:30:04.090555 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.090426    2547 scope.go:117] "RemoveContainer" containerID="87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3"
Jul 30 20:30:04.090555 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.090501    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:30:04.100316 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.100296    2547 scope.go:117] "RemoveContainer" containerID="87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3"
Jul 30 20:30:04.100692 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:30:04.100668    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3\": container with ID starting with 87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3 not found: ID does not exist" containerID="87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3"
Jul 30 20:30:04.100786 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.100717    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3"} err="failed to get container status \"87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3\": rpc error: code = NotFound desc = could not find container \"87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3\": container with ID starting with 87a02d7899e54b2429984e98540701902356932a400f94072c079cea3eb0c0b3 not found: ID does not exist"
Jul 30 20:30:04.108820 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.108795    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75757fc68b-5gs8h"]
Jul 30 20:30:04.112378 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:04.112352    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75757fc68b-5gs8h"]
Jul 30 20:30:06.073518 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:06.073477    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e49f3bf5-4164-4228-bdb2-5145951d33e0" path="/var/lib/kubelet/pods/e49f3bf5-4164-4228-bdb2-5145951d33e0/volumes"
Jul 30 20:30:06.837335 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:06.837301    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:30:06.840164 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:06.840132    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bfaea5d-a4b2-42a0-889d-29b157f75146-metrics-tls\") pod \"dns-default-tjtsk\" (UID: \"4bfaea5d-a4b2-42a0-889d-29b157f75146\") " pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:30:07.094410 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:07.094339    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tn7n7\""
Jul 30 20:30:07.101958 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:07.101930    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:30:07.252773 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:07.252742    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tjtsk"]
Jul 30 20:30:07.255758 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:30:07.255728    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bfaea5d_a4b2_42a0_889d_29b157f75146.slice/crio-44e5f4ac19f635c5333eca6d4efcce9ad22f1c459a95e8f057eadfedd5240033 WatchSource:0}: Error finding container 44e5f4ac19f635c5333eca6d4efcce9ad22f1c459a95e8f057eadfedd5240033: Status 404 returned error can't find the container with id 44e5f4ac19f635c5333eca6d4efcce9ad22f1c459a95e8f057eadfedd5240033
Jul 30 20:30:08.110593 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:08.110523    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tjtsk" event={"ID":"4bfaea5d-a4b2-42a0-889d-29b157f75146","Type":"ContainerStarted","Data":"44e5f4ac19f635c5333eca6d4efcce9ad22f1c459a95e8f057eadfedd5240033"}
Jul 30 20:30:10.121592 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:10.121551    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tjtsk" event={"ID":"4bfaea5d-a4b2-42a0-889d-29b157f75146","Type":"ContainerStarted","Data":"4e4d0246c0f5dcd6e2bafc025344a89aa40a90415023c1bd65c299bdc3e0d50a"}
Jul 30 20:30:10.122072 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:10.121620    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tjtsk" event={"ID":"4bfaea5d-a4b2-42a0-889d-29b157f75146","Type":"ContainerStarted","Data":"3d906003f494dc496a38c834c7397e6095cd2b28fa58b382ffeb4190a4233d21"}
Jul 30 20:30:10.122072 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:10.121836    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:30:10.139858 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:10.139787    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tjtsk" podStartSLOduration=252.261810804 podStartE2EDuration="4m14.139771508s" totalImagesPullingTime="1.877960704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:30:10.139390559 +0000 UTC m=+288.600349370" watchObservedRunningTime="2026-07-30 20:30:10.139771508 +0000 UTC m=+288.600730319"
Jul 30 20:30:16.257391 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:16.257344    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:30:16.258009 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:16.257449    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:30:17.346888 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.346843    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-676c5597f-x5phx"]
Jul 30 20:30:17.347333 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.347304    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49f3bf5-4164-4228-bdb2-5145951d33e0" containerName="registry"
Jul 30 20:30:17.347333 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.347319    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49f3bf5-4164-4228-bdb2-5145951d33e0" containerName="registry"
Jul 30 20:30:17.347421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.347389    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49f3bf5-4164-4228-bdb2-5145951d33e0" containerName="registry"
Jul 30 20:30:17.351789 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.351763    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.356124 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.356102    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 20:30:17.357369 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.357348    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 20:30:17.357369 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.357362    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 20:30:17.357978 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.357786    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 20:30:17.357978 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.357808    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 20:30:17.357978 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.357931    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mf675\""
Jul 30 20:30:17.362223 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.362203    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 20:30:17.369368 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.369349    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-676c5597f-x5phx"]
Jul 30 20:30:17.438925 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.438889    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-serving-cert\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.438925 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.438930    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkl6q\" (UniqueName: \"kubernetes.io/projected/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-kube-api-access-xkl6q\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.439192 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.438952    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-service-ca\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.439192 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.439078    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-oauth-config\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.439192 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.439113    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-config\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.439342 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.439214    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-trusted-ca-bundle\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.439342 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.439265    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-oauth-serving-cert\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.540417 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.540380    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-oauth-serving-cert\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.540614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.540461    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-serving-cert\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.540614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.540488    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkl6q\" (UniqueName: \"kubernetes.io/projected/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-kube-api-access-xkl6q\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.540614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.540508    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-service-ca\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.540614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.540557    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-oauth-config\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.540614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.540582    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-config\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.540917 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.540652    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-trusted-ca-bundle\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.541547 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.541283    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-service-ca\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.541547 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.541283    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-oauth-serving-cert\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.541547 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.541491    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-trusted-ca-bundle\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.541781 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.541753    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-config\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.543446 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.543419    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-serving-cert\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.543567 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.543531    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-oauth-config\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.552559 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.552539    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkl6q\" (UniqueName: \"kubernetes.io/projected/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-kube-api-access-xkl6q\") pod \"console-676c5597f-x5phx\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") " pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:17.663460 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:17.663361    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:19.672134 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:19.672109    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-676c5597f-x5phx"]
Jul 30 20:30:19.675584 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:30:19.675561    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd3f16ce_c1ef_41ec_9db9_784f8979ceac.slice/crio-5f404ea6de77b660988532583a98e36f1467b1fa51ac93b470ee5290e2884421 WatchSource:0}: Error finding container 5f404ea6de77b660988532583a98e36f1467b1fa51ac93b470ee5290e2884421: Status 404 returned error can't find the container with id 5f404ea6de77b660988532583a98e36f1467b1fa51ac93b470ee5290e2884421
Jul 30 20:30:20.130199 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:20.130160    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tjtsk"
Jul 30 20:30:20.168738 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:20.168678    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676c5597f-x5phx" event={"ID":"cd3f16ce-c1ef-41ec-9db9-784f8979ceac","Type":"ContainerStarted","Data":"5f404ea6de77b660988532583a98e36f1467b1fa51ac93b470ee5290e2884421"}
Jul 30 20:30:20.170855 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:20.170820    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-gdngc" event={"ID":"cba71dcb-757a-4991-8831-6899b4e8876a","Type":"ContainerStarted","Data":"b5e73e54f7cc62d8b4b9c0f6868bf8231b7a8ba47549718099f51b5317a466d9"}
Jul 30 20:30:20.171118 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:20.171095    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-gdngc"
Jul 30 20:30:20.185442 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:20.185410    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-gdngc"
Jul 30 20:30:20.189974 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:20.189921    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-gdngc" podStartSLOduration=1.794493399 podStartE2EDuration="18.189901625s" totalImagesPullingTime="16.395408226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:30:20.18864982 +0000 UTC m=+298.649608633" watchObservedRunningTime="2026-07-30 20:30:20.189901625 +0000 UTC m=+298.650860437"
Jul 30 20:30:21.956468 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:21.956435    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:30:21.957334 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:21.957232    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:30:21.970518 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:21.970491    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:30:21.970798 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:21.970778    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:30:21.983670 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:21.983647    2547 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 20:30:25.201920 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:25.201864    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676c5597f-x5phx" event={"ID":"cd3f16ce-c1ef-41ec-9db9-784f8979ceac","Type":"ContainerStarted","Data":"cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d"}
Jul 30 20:30:25.228789 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:25.228674    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-676c5597f-x5phx" podStartSLOduration=3.699140904 podStartE2EDuration="8.228645127s" totalImagesPullingTime="4.529504223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:30:25.227518086 +0000 UTC m=+303.688476892" watchObservedRunningTime="2026-07-30 20:30:25.228645127 +0000 UTC m=+303.689603940"
Jul 30 20:30:27.664136 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:27.664090    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:27.664735 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:27.664175    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:27.669899 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:27.669876    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:28.221186 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:28.221146    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:30:36.263035 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:36.263005    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:30:36.266923 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:30:36.266902    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-65f7ff9d9b-bghbb"
Jul 30 20:31:12.029915 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.029878    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 20:31:12.030539 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.030298    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="alertmanager" containerID="cri-o://878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49" gracePeriod=120
Jul 30 20:31:12.030539 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.030342    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy-metric" containerID="cri-o://e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795" gracePeriod=120
Jul 30 20:31:12.030539 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.030384    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy-web" containerID="cri-o://2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d" gracePeriod=120
Jul 30 20:31:12.030539 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.030399    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy" containerID="cri-o://bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb" gracePeriod=120
Jul 30 20:31:12.030539 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.030433    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="config-reloader" containerID="cri-o://e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970" gracePeriod=120
Jul 30 20:31:12.030539 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.030397    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="prom-label-proxy" containerID="cri-o://59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4" gracePeriod=120
Jul 30 20:31:12.388824 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.388793    2547 generic.go:356] "Generic (PLEG): container finished" podID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerID="59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4" exitCode=0
Jul 30 20:31:12.388824 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.388820    2547 generic.go:356] "Generic (PLEG): container finished" podID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerID="bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb" exitCode=0
Jul 30 20:31:12.388824 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.388829    2547 generic.go:356] "Generic (PLEG): container finished" podID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerID="e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970" exitCode=0
Jul 30 20:31:12.389049 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.388838    2547 generic.go:356] "Generic (PLEG): container finished" podID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerID="878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49" exitCode=0
Jul 30 20:31:12.389049 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.388868    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerDied","Data":"59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4"}
Jul 30 20:31:12.389049 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.388934    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerDied","Data":"bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb"}
Jul 30 20:31:12.389049 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.388962    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerDied","Data":"e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970"}
Jul 30 20:31:12.389049 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:12.388990    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerDied","Data":"878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49"}
Jul 30 20:31:13.373186 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.373165    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.395624 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.395596    2547 generic.go:356] "Generic (PLEG): container finished" podID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerID="e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795" exitCode=0
Jul 30 20:31:13.395624 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.395617    2547 generic.go:356] "Generic (PLEG): container finished" podID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerID="2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d" exitCode=0
Jul 30 20:31:13.395787 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.395649    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerDied","Data":"e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795"}
Jul 30 20:31:13.395787 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.395661    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.395787 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.395716    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerDied","Data":"2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d"}
Jul 30 20:31:13.395787 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.395743    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"de08378f-1a11-480b-83c7-71ea2ce9ece2","Type":"ContainerDied","Data":"9fc6bb5813aeb852ba2a09d0ac09bd2b6984871e63ebb3b729d3cc432f400f28"}
Jul 30 20:31:13.395787 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.395772    2547 scope.go:117] "RemoveContainer" containerID="59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4"
Jul 30 20:31:13.403768 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.403745    2547 scope.go:117] "RemoveContainer" containerID="e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795"
Jul 30 20:31:13.413172 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.413141    2547 scope.go:117] "RemoveContainer" containerID="bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb"
Jul 30 20:31:13.421198 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.421177    2547 scope.go:117] "RemoveContainer" containerID="2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d"
Jul 30 20:31:13.428247 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.428216    2547 scope.go:117] "RemoveContainer" containerID="e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970"
Jul 30 20:31:13.435325 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.435305    2547 scope.go:117] "RemoveContainer" containerID="878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49"
Jul 30 20:31:13.442387 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.442368    2547 scope.go:117] "RemoveContainer" containerID="bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc"
Jul 30 20:31:13.449271 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.449251    2547 scope.go:117] "RemoveContainer" containerID="59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4"
Jul 30 20:31:13.449544 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:31:13.449519    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4\": container with ID starting with 59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4 not found: ID does not exist" containerID="59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4"
Jul 30 20:31:13.449620 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.449554    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4"} err="failed to get container status \"59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4\": rpc error: code = NotFound desc = could not find container \"59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4\": container with ID starting with 59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4 not found: ID does not exist"
Jul 30 20:31:13.449620 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.449582    2547 scope.go:117] "RemoveContainer" containerID="e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795"
Jul 30 20:31:13.449845 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:31:13.449818    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795\": container with ID starting with e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795 not found: ID does not exist" containerID="e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795"
Jul 30 20:31:13.449946 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.449853    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795"} err="failed to get container status \"e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795\": rpc error: code = NotFound desc = could not find container \"e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795\": container with ID starting with e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795 not found: ID does not exist"
Jul 30 20:31:13.449946 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.449878    2547 scope.go:117] "RemoveContainer" containerID="bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb"
Jul 30 20:31:13.450142 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:31:13.450125    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb\": container with ID starting with bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb not found: ID does not exist" containerID="bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb"
Jul 30 20:31:13.450192 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.450151    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb"} err="failed to get container status \"bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb\": rpc error: code = NotFound desc = could not find container \"bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb\": container with ID starting with bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb not found: ID does not exist"
Jul 30 20:31:13.450192 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.450169    2547 scope.go:117] "RemoveContainer" containerID="2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d"
Jul 30 20:31:13.450421 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:31:13.450404    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d\": container with ID starting with 2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d not found: ID does not exist" containerID="2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d"
Jul 30 20:31:13.450471 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.450428    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d"} err="failed to get container status \"2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d\": rpc error: code = NotFound desc = could not find container \"2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d\": container with ID starting with 2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d not found: ID does not exist"
Jul 30 20:31:13.450471 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.450444    2547 scope.go:117] "RemoveContainer" containerID="e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970"
Jul 30 20:31:13.450686 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:31:13.450667    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970\": container with ID starting with e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970 not found: ID does not exist" containerID="e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970"
Jul 30 20:31:13.450759 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.450694    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970"} err="failed to get container status \"e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970\": rpc error: code = NotFound desc = could not find container \"e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970\": container with ID starting with e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970 not found: ID does not exist"
Jul 30 20:31:13.450800 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.450765    2547 scope.go:117] "RemoveContainer" containerID="878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49"
Jul 30 20:31:13.451002 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:31:13.450981    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49\": container with ID starting with 878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49 not found: ID does not exist" containerID="878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49"
Jul 30 20:31:13.451067 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451007    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49"} err="failed to get container status \"878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49\": rpc error: code = NotFound desc = could not find container \"878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49\": container with ID starting with 878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49 not found: ID does not exist"
Jul 30 20:31:13.451067 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451024    2547 scope.go:117] "RemoveContainer" containerID="bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc"
Jul 30 20:31:13.451255 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:31:13.451238    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc\": container with ID starting with bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc not found: ID does not exist" containerID="bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc"
Jul 30 20:31:13.451315 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451260    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc"} err="failed to get container status \"bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc\": rpc error: code = NotFound desc = could not find container \"bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc\": container with ID starting with bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc not found: ID does not exist"
Jul 30 20:31:13.451315 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451274    2547 scope.go:117] "RemoveContainer" containerID="59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4"
Jul 30 20:31:13.451481 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451453    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4"} err="failed to get container status \"59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4\": rpc error: code = NotFound desc = could not find container \"59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4\": container with ID starting with 59e6d1aeddbf746c6554c6e0ac0fa848d2838b7f3394bd4d47c1be15ed7a70f4 not found: ID does not exist"
Jul 30 20:31:13.451481 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451481    2547 scope.go:117] "RemoveContainer" containerID="e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795"
Jul 30 20:31:13.451747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451729    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795"} err="failed to get container status \"e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795\": rpc error: code = NotFound desc = could not find container \"e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795\": container with ID starting with e478f0cf83f01d9cbd744f8a6809f4c4441630a1dc4d3666edb9382329ae8795 not found: ID does not exist"
Jul 30 20:31:13.451811 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451747    2547 scope.go:117] "RemoveContainer" containerID="bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb"
Jul 30 20:31:13.451988 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451972    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb"} err="failed to get container status \"bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb\": rpc error: code = NotFound desc = could not find container \"bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb\": container with ID starting with bc60a4546d261f989ab4d5bddf6f29fa71fd16d74b0a4d061eeda33818139bcb not found: ID does not exist"
Jul 30 20:31:13.452039 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.451989    2547 scope.go:117] "RemoveContainer" containerID="2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d"
Jul 30 20:31:13.452212 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.452195    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d"} err="failed to get container status \"2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d\": rpc error: code = NotFound desc = could not find container \"2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d\": container with ID starting with 2e07258165dfecc3be9f86061e4e0a523e576a475de6ffadea5e3173f21b6d7d not found: ID does not exist"
Jul 30 20:31:13.452212 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.452212    2547 scope.go:117] "RemoveContainer" containerID="e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970"
Jul 30 20:31:13.452439 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.452417    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970"} err="failed to get container status \"e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970\": rpc error: code = NotFound desc = could not find container \"e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970\": container with ID starting with e81666a3bc2975c77cc743ed3c200b5b5d4b5029431a40547e998c21e9315970 not found: ID does not exist"
Jul 30 20:31:13.452488 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.452442    2547 scope.go:117] "RemoveContainer" containerID="878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49"
Jul 30 20:31:13.452685 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.452666    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49"} err="failed to get container status \"878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49\": rpc error: code = NotFound desc = could not find container \"878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49\": container with ID starting with 878ecbd9dcf1df29b0697416f8f49a12a6da4f648c4fcff07fc2eb2d8d4ebd49 not found: ID does not exist"
Jul 30 20:31:13.452755 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.452686    2547 scope.go:117] "RemoveContainer" containerID="bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc"
Jul 30 20:31:13.452959 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.452931    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc"} err="failed to get container status \"bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc\": rpc error: code = NotFound desc = could not find container \"bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc\": container with ID starting with bf790588919fc64bfe85334e1cc3b4ecb5168e865e934eebe97ba48485b915cc not found: ID does not exist"
Jul 30 20:31:13.504842 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.504819    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5kxz\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-kube-api-access-x5kxz\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.504947 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.504862    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-trusted-ca-bundle\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.504947 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.504881    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-cluster-tls-config\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.504947 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.504907    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.504947 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.504930    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-web\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.505130 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.504957    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-volume\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.505130 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505073    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-out\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.505130 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505112    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-metrics-client-ca\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.505276 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505149    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-main-db\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.505276 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505182    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-web-config\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.505276 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505237    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-tls-assets\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.505414 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505279    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-main-tls\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.505414 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505298    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:31:13.505414 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505317    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"de08378f-1a11-480b-83c7-71ea2ce9ece2\" (UID: \"de08378f-1a11-480b-83c7-71ea2ce9ece2\") "
Jul 30 20:31:13.505577 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505560    2547 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.505924 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.505620    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:31:13.506464 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.506435    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:31:13.508228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.508032    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-kube-api-access-x5kxz" (OuterVolumeSpecName: "kube-api-access-x5kxz") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "kube-api-access-x5kxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:31:13.508335 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.508288    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-volume" (OuterVolumeSpecName: "config-volume") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:31:13.508929 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.508892    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:31:13.508929 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.508915    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:31:13.509084 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.509023    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:31:13.509146 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.509124    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-out" (OuterVolumeSpecName: "config-out") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:31:13.509458 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.509427    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:31:13.509806 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.509781    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:31:13.512617 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.512484    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:31:13.519350 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.519330    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-web-config" (OuterVolumeSpecName: "web-config") pod "de08378f-1a11-480b-83c7-71ea2ce9ece2" (UID: "de08378f-1a11-480b-83c7-71ea2ce9ece2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:31:13.606827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606754    2547 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.606827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606788    2547 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.606827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606799    2547 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.606827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606807    2547 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-config-out\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.606827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606817    2547 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de08378f-1a11-480b-83c7-71ea2ce9ece2-metrics-client-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.606827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606828    2547 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/de08378f-1a11-480b-83c7-71ea2ce9ece2-alertmanager-main-db\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.606827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606838    2547 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-web-config\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.607102 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606846    2547 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-tls-assets\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.607102 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606855    2547 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-main-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.607102 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606862    2547 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.607102 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606871    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5kxz\" (UniqueName: \"kubernetes.io/projected/de08378f-1a11-480b-83c7-71ea2ce9ece2-kube-api-access-x5kxz\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.607102 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.606878    2547 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/de08378f-1a11-480b-83c7-71ea2ce9ece2-cluster-tls-config\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:13.721621 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.721591    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 20:31:13.726274 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.726246    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 20:31:13.751024 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.750999    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 20:31:13.751264 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751253    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy"
Jul 30 20:31:13.751320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751266    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy"
Jul 30 20:31:13.751320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751279    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="init-config-reloader"
Jul 30 20:31:13.751320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751284    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="init-config-reloader"
Jul 30 20:31:13.751320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751291    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy-metric"
Jul 30 20:31:13.751320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751297    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy-metric"
Jul 30 20:31:13.751320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751304    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="config-reloader"
Jul 30 20:31:13.751320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751309    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="config-reloader"
Jul 30 20:31:13.751320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751316    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy-web"
Jul 30 20:31:13.751320 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751321    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy-web"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751332    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="prom-label-proxy"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751337    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="prom-label-proxy"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751345    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="alertmanager"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751349    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="alertmanager"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751388    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="config-reloader"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751394    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy-metric"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751399    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="prom-label-proxy"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751404    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751410    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="kube-rbac-proxy-web"
Jul 30 20:31:13.751614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.751415    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" containerName="alertmanager"
Jul 30 20:31:13.755108 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.755090    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.757926 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.757905    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 30 20:31:13.758036 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.757988    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 30 20:31:13.758036 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.757993    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 30 20:31:13.758148 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.758127    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 30 20:31:13.758268 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.758245    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vtgvg\""
Jul 30 20:31:13.758348 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.758292    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 30 20:31:13.758408 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.758347    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 30 20:31:13.758527 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.758512    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 30 20:31:13.759230 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.759207    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 30 20:31:13.763633 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.763616    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 30 20:31:13.773786 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.773763    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 20:31:13.909581 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909487    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/905745e6-0eb4-4fb0-bba1-c638349052c4-config-out\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909581 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909540    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/905745e6-0eb4-4fb0-bba1-c638349052c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909581 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909563    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-web-config\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909581 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909583    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/905745e6-0eb4-4fb0-bba1-c638349052c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909873 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909611    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909873 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909628    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909873 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909739    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909873 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909772    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkll8\" (UniqueName: \"kubernetes.io/projected/905745e6-0eb4-4fb0-bba1-c638349052c4-kube-api-access-rkll8\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909873 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909790    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909873 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909804    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/905745e6-0eb4-4fb0-bba1-c638349052c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909873 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909822    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.909873 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909870    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:13.910111 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:13.909905    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/905745e6-0eb4-4fb0-bba1-c638349052c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.010763 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.010731    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/905745e6-0eb4-4fb0-bba1-c638349052c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.010864 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.010773    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.010864 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.010792    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.010864 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.010843    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.010992 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.010870    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkll8\" (UniqueName: \"kubernetes.io/projected/905745e6-0eb4-4fb0-bba1-c638349052c4-kube-api-access-rkll8\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.010992 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.010893    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.011129 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.011017    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/905745e6-0eb4-4fb0-bba1-c638349052c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.011129 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.011062    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.011129 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.011095    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.011263 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.011128    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/905745e6-0eb4-4fb0-bba1-c638349052c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.011263 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.011161    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/905745e6-0eb4-4fb0-bba1-c638349052c4-config-out\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.011263 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.011200    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/905745e6-0eb4-4fb0-bba1-c638349052c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.011263 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.011233    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-web-config\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.011565 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.011542    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/905745e6-0eb4-4fb0-bba1-c638349052c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.014402 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.014223    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.014402 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.014237    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.014402 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.014243    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.014402 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.014329    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.014402 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.014361    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/905745e6-0eb4-4fb0-bba1-c638349052c4-config-out\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.014617 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.014424    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.014617 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.014456    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.014617 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.014517    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/905745e6-0eb4-4fb0-bba1-c638349052c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.014733 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.014679    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/905745e6-0eb4-4fb0-bba1-c638349052c4-web-config\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.015147 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.015126    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/905745e6-0eb4-4fb0-bba1-c638349052c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.016151 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.016134    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/905745e6-0eb4-4fb0-bba1-c638349052c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.020456 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.020440    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkll8\" (UniqueName: \"kubernetes.io/projected/905745e6-0eb4-4fb0-bba1-c638349052c4-kube-api-access-rkll8\") pod \"alertmanager-main-0\" (UID: \"905745e6-0eb4-4fb0-bba1-c638349052c4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.064377 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.064344    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 20:31:14.071075 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.071050    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de08378f-1a11-480b-83c7-71ea2ce9ece2" path="/var/lib/kubelet/pods/de08378f-1a11-480b-83c7-71ea2ce9ece2/volumes"
Jul 30 20:31:14.219653 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.219619    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 20:31:14.223001 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:31:14.222957    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod905745e6_0eb4_4fb0_bba1_c638349052c4.slice/crio-611f2be439dbd5e42f4365b6c5a951faeaf24ea64ce28d73e4104c79d018c980 WatchSource:0}: Error finding container 611f2be439dbd5e42f4365b6c5a951faeaf24ea64ce28d73e4104c79d018c980: Status 404 returned error can't find the container with id 611f2be439dbd5e42f4365b6c5a951faeaf24ea64ce28d73e4104c79d018c980
Jul 30 20:31:14.401334 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.401305    2547 generic.go:356] "Generic (PLEG): container finished" podID="905745e6-0eb4-4fb0-bba1-c638349052c4" containerID="4f0b263ce2a47847a4be349303f1a6de861fcfbed2d63bf4ecabe380889abfb3" exitCode=0
Jul 30 20:31:14.401779 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.401396    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"905745e6-0eb4-4fb0-bba1-c638349052c4","Type":"ContainerDied","Data":"4f0b263ce2a47847a4be349303f1a6de861fcfbed2d63bf4ecabe380889abfb3"}
Jul 30 20:31:14.401779 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:14.401468    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"905745e6-0eb4-4fb0-bba1-c638349052c4","Type":"ContainerStarted","Data":"611f2be439dbd5e42f4365b6c5a951faeaf24ea64ce28d73e4104c79d018c980"}
Jul 30 20:31:15.409641 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:15.409603    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"905745e6-0eb4-4fb0-bba1-c638349052c4","Type":"ContainerStarted","Data":"db60fcff6bdd60893510e21f7f6ecd1769debb41a05364d970d8406ef756caee"}
Jul 30 20:31:15.410110 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:15.409713    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"905745e6-0eb4-4fb0-bba1-c638349052c4","Type":"ContainerStarted","Data":"795f2778911f460a53f97cbd2af2f5866e65158635a7e01fdf943e74e3f5b8ac"}
Jul 30 20:31:15.410110 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:15.409745    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"905745e6-0eb4-4fb0-bba1-c638349052c4","Type":"ContainerStarted","Data":"9888be79c40ee1ce94b7f9c3396a88e805bfc2fc6808ee9bb78d84e3ac7d1776"}
Jul 30 20:31:15.410110 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:15.409768    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"905745e6-0eb4-4fb0-bba1-c638349052c4","Type":"ContainerStarted","Data":"c626e299177e866b57691ea9b527af903fc55122c1b52f133f2914260f32e490"}
Jul 30 20:31:15.410110 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:15.409790    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"905745e6-0eb4-4fb0-bba1-c638349052c4","Type":"ContainerStarted","Data":"37f512c7c6dd97adde6773478da2e92eb349bc2a3abd8f567b59216f19c7b7e3"}
Jul 30 20:31:15.410110 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:15.409812    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"905745e6-0eb4-4fb0-bba1-c638349052c4","Type":"ContainerStarted","Data":"48d2ec6d4ba8f6d37be6c3784990113f66284317355e4c401e306cfaaea2637f"}
Jul 30 20:31:15.440337 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:15.440290    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.423877288 podStartE2EDuration="2.440276688s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.3994ms" isStatefulPod=false podCreationTimestamp="2026-07-30 20:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:31:15.437871047 +0000 UTC m=+353.898829853" watchObservedRunningTime="2026-07-30 20:31:15.440276688 +0000 UTC m=+353.901235501"
Jul 30 20:31:30.087627 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:30.087594    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-676c5597f-x5phx"]
Jul 30 20:31:55.114197 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.114135    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-676c5597f-x5phx" podUID="cd3f16ce-c1ef-41ec-9db9-784f8979ceac" containerName="console" containerID="cri-o://cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d" gracePeriod=15
Jul 30 20:31:55.354430 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.354407    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:31:55.422299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422240    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-oauth-serving-cert\") pod \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") "
Jul 30 20:31:55.422299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422268    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-config\") pod \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") "
Jul 30 20:31:55.422535 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422302    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-oauth-config\") pod \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") "
Jul 30 20:31:55.422535 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422322    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-service-ca\") pod \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") "
Jul 30 20:31:55.422535 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422338    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-trusted-ca-bundle\") pod \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") "
Jul 30 20:31:55.422535 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422357    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-serving-cert\") pod \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") "
Jul 30 20:31:55.422535 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422421    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkl6q\" (UniqueName: \"kubernetes.io/projected/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-kube-api-access-xkl6q\") pod \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\" (UID: \"cd3f16ce-c1ef-41ec-9db9-784f8979ceac\") "
Jul 30 20:31:55.422828 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422662    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-config" (OuterVolumeSpecName: "console-config") pod "cd3f16ce-c1ef-41ec-9db9-784f8979ceac" (UID: "cd3f16ce-c1ef-41ec-9db9-784f8979ceac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:31:55.422828 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422759    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cd3f16ce-c1ef-41ec-9db9-784f8979ceac" (UID: "cd3f16ce-c1ef-41ec-9db9-784f8979ceac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:31:55.422828 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.422783    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-service-ca" (OuterVolumeSpecName: "service-ca") pod "cd3f16ce-c1ef-41ec-9db9-784f8979ceac" (UID: "cd3f16ce-c1ef-41ec-9db9-784f8979ceac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:31:55.423233 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.423207    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cd3f16ce-c1ef-41ec-9db9-784f8979ceac" (UID: "cd3f16ce-c1ef-41ec-9db9-784f8979ceac"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:31:55.424748 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.424719    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cd3f16ce-c1ef-41ec-9db9-784f8979ceac" (UID: "cd3f16ce-c1ef-41ec-9db9-784f8979ceac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:31:55.424837 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.424751    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cd3f16ce-c1ef-41ec-9db9-784f8979ceac" (UID: "cd3f16ce-c1ef-41ec-9db9-784f8979ceac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:31:55.424837 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.424763    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-kube-api-access-xkl6q" (OuterVolumeSpecName: "kube-api-access-xkl6q") pod "cd3f16ce-c1ef-41ec-9db9-784f8979ceac" (UID: "cd3f16ce-c1ef-41ec-9db9-784f8979ceac"). InnerVolumeSpecName "kube-api-access-xkl6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:31:55.523432 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.523408    2547 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-oauth-serving-cert\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:55.523432 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.523430    2547 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-config\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:55.523564 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.523440    2547 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-oauth-config\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:55.523564 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.523450    2547 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:55.523564 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.523458    2547 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-trusted-ca-bundle\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:55.523564 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.523466    2547 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-console-serving-cert\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:55.523564 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.523474    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkl6q\" (UniqueName: \"kubernetes.io/projected/cd3f16ce-c1ef-41ec-9db9-784f8979ceac-kube-api-access-xkl6q\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:31:55.553803 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.553776    2547 generic.go:356] "Generic (PLEG): container finished" podID="cd3f16ce-c1ef-41ec-9db9-784f8979ceac" containerID="cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d" exitCode=0
Jul 30 20:31:55.553879 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.553826    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676c5597f-x5phx"
Jul 30 20:31:55.553879 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.553848    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676c5597f-x5phx" event={"ID":"cd3f16ce-c1ef-41ec-9db9-784f8979ceac","Type":"ContainerDied","Data":"cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d"}
Jul 30 20:31:55.553960 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.553924    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676c5597f-x5phx" event={"ID":"cd3f16ce-c1ef-41ec-9db9-784f8979ceac","Type":"ContainerDied","Data":"5f404ea6de77b660988532583a98e36f1467b1fa51ac93b470ee5290e2884421"}
Jul 30 20:31:55.553960 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.553949    2547 scope.go:117] "RemoveContainer" containerID="cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d"
Jul 30 20:31:55.561813 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.561672    2547 scope.go:117] "RemoveContainer" containerID="cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d"
Jul 30 20:31:55.562019 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:31:55.561996    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d\": container with ID starting with cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d not found: ID does not exist" containerID="cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d"
Jul 30 20:31:55.562094 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.562034    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d"} err="failed to get container status \"cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d\": rpc error: code = NotFound desc = could not find container \"cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d\": container with ID starting with cd268a0a6237e6c10096811862ca0c285997caed7f8b49af1946fd3c2936863d not found: ID does not exist"
Jul 30 20:31:55.574073 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.574053    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-676c5597f-x5phx"]
Jul 30 20:31:55.579808 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:55.579787    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-676c5597f-x5phx"]
Jul 30 20:31:56.070374 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:31:56.070347    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd3f16ce-c1ef-41ec-9db9-784f8979ceac" path="/var/lib/kubelet/pods/cd3f16ce-c1ef-41ec-9db9-784f8979ceac/volumes"
Jul 30 20:32:57.881772 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:57.881648    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4555d"]
Jul 30 20:32:57.882385 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:57.882128    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd3f16ce-c1ef-41ec-9db9-784f8979ceac" containerName="console"
Jul 30 20:32:57.882385 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:57.882150    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd3f16ce-c1ef-41ec-9db9-784f8979ceac" containerName="console"
Jul 30 20:32:57.882385 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:57.882230    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd3f16ce-c1ef-41ec-9db9-784f8979ceac" containerName="console"
Jul 30 20:32:57.884428 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:57.884402    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d"
Jul 30 20:32:57.887412 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:57.887391    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lm9vz\""
Jul 30 20:32:57.888462 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:57.888441    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 30 20:32:57.888514 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:57.888471    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 30 20:32:57.894734 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:57.894686    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4555d"]
Jul 30 20:32:58.042739 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.042691    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94jm2\" (UniqueName: \"kubernetes.io/projected/a33d14e0-8219-4990-9660-e84b2ef30ed4-kube-api-access-94jm2\") pod \"cert-manager-cainjector-559cdd8c87-4555d\" (UID: \"a33d14e0-8219-4990-9660-e84b2ef30ed4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d"
Jul 30 20:32:58.042880 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.042760    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a33d14e0-8219-4990-9660-e84b2ef30ed4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4555d\" (UID: \"a33d14e0-8219-4990-9660-e84b2ef30ed4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d"
Jul 30 20:32:58.143291 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.143228    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94jm2\" (UniqueName: \"kubernetes.io/projected/a33d14e0-8219-4990-9660-e84b2ef30ed4-kube-api-access-94jm2\") pod \"cert-manager-cainjector-559cdd8c87-4555d\" (UID: \"a33d14e0-8219-4990-9660-e84b2ef30ed4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d"
Jul 30 20:32:58.143291 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.143268    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a33d14e0-8219-4990-9660-e84b2ef30ed4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4555d\" (UID: \"a33d14e0-8219-4990-9660-e84b2ef30ed4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d"
Jul 30 20:32:58.153591 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.153567    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94jm2\" (UniqueName: \"kubernetes.io/projected/a33d14e0-8219-4990-9660-e84b2ef30ed4-kube-api-access-94jm2\") pod \"cert-manager-cainjector-559cdd8c87-4555d\" (UID: \"a33d14e0-8219-4990-9660-e84b2ef30ed4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d"
Jul 30 20:32:58.153692 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.153645    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a33d14e0-8219-4990-9660-e84b2ef30ed4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4555d\" (UID: \"a33d14e0-8219-4990-9660-e84b2ef30ed4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d"
Jul 30 20:32:58.206656 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.206635    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d"
Jul 30 20:32:58.333149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.333119    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4555d"]
Jul 30 20:32:58.335742 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:32:58.335714    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda33d14e0_8219_4990_9660_e84b2ef30ed4.slice/crio-e042213e99629f52ba49e4fabc19f19e27be3f086c8ce63c66fb78b0be59010a WatchSource:0}: Error finding container e042213e99629f52ba49e4fabc19f19e27be3f086c8ce63c66fb78b0be59010a: Status 404 returned error can't find the container with id e042213e99629f52ba49e4fabc19f19e27be3f086c8ce63c66fb78b0be59010a
Jul 30 20:32:58.337806 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.337786    2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:32:58.797196 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:32:58.797159    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d" event={"ID":"a33d14e0-8219-4990-9660-e84b2ef30ed4","Type":"ContainerStarted","Data":"e042213e99629f52ba49e4fabc19f19e27be3f086c8ce63c66fb78b0be59010a"}
Jul 30 20:33:01.811258 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:01.811223    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d" event={"ID":"a33d14e0-8219-4990-9660-e84b2ef30ed4","Type":"ContainerStarted","Data":"d19981392839da1d962547e82390fb484aecf94f8a2d64463373511c236ad981"}
Jul 30 20:33:01.828291 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:01.828216    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4555d" podStartSLOduration=2.107053584 podStartE2EDuration="4.828199302s" totalImagesPullingTime="2.721145718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:33:01.827962836 +0000 UTC m=+460.288921649" watchObservedRunningTime="2026-07-30 20:33:01.828199302 +0000 UTC m=+460.289158118"
Jul 30 20:33:16.185347 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.185310    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9k2t5"]
Jul 30 20:33:16.189542 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.189513    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9k2t5"
Jul 30 20:33:16.192207 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.192179    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7ggc7\""
Jul 30 20:33:16.198582 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.198555    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9k2t5"]
Jul 30 20:33:16.368273 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.368248    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksmjc\" (UniqueName: \"kubernetes.io/projected/73305c5b-9360-4c08-af2c-99f3ca6fcded-kube-api-access-ksmjc\") pod \"cert-manager-55b4d954d5-9k2t5\" (UID: \"73305c5b-9360-4c08-af2c-99f3ca6fcded\") " pod="cert-manager/cert-manager-55b4d954d5-9k2t5"
Jul 30 20:33:16.368388 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.368281    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73305c5b-9360-4c08-af2c-99f3ca6fcded-bound-sa-token\") pod \"cert-manager-55b4d954d5-9k2t5\" (UID: \"73305c5b-9360-4c08-af2c-99f3ca6fcded\") " pod="cert-manager/cert-manager-55b4d954d5-9k2t5"
Jul 30 20:33:16.469423 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.469339    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksmjc\" (UniqueName: \"kubernetes.io/projected/73305c5b-9360-4c08-af2c-99f3ca6fcded-kube-api-access-ksmjc\") pod \"cert-manager-55b4d954d5-9k2t5\" (UID: \"73305c5b-9360-4c08-af2c-99f3ca6fcded\") " pod="cert-manager/cert-manager-55b4d954d5-9k2t5"
Jul 30 20:33:16.469423 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.469407    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73305c5b-9360-4c08-af2c-99f3ca6fcded-bound-sa-token\") pod \"cert-manager-55b4d954d5-9k2t5\" (UID: \"73305c5b-9360-4c08-af2c-99f3ca6fcded\") " pod="cert-manager/cert-manager-55b4d954d5-9k2t5"
Jul 30 20:33:16.478135 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.478104    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73305c5b-9360-4c08-af2c-99f3ca6fcded-bound-sa-token\") pod \"cert-manager-55b4d954d5-9k2t5\" (UID: \"73305c5b-9360-4c08-af2c-99f3ca6fcded\") " pod="cert-manager/cert-manager-55b4d954d5-9k2t5"
Jul 30 20:33:16.478358 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.478330    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksmjc\" (UniqueName: \"kubernetes.io/projected/73305c5b-9360-4c08-af2c-99f3ca6fcded-kube-api-access-ksmjc\") pod \"cert-manager-55b4d954d5-9k2t5\" (UID: \"73305c5b-9360-4c08-af2c-99f3ca6fcded\") " pod="cert-manager/cert-manager-55b4d954d5-9k2t5"
Jul 30 20:33:16.499785 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.499744    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9k2t5"
Jul 30 20:33:16.631937 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.631907    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9k2t5"]
Jul 30 20:33:16.635360 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:33:16.635337    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73305c5b_9360_4c08_af2c_99f3ca6fcded.slice/crio-c87d2acd49e0e95a247de1ffa8cd346f921879634b71751fc34321eda2e61d5d WatchSource:0}: Error finding container c87d2acd49e0e95a247de1ffa8cd346f921879634b71751fc34321eda2e61d5d: Status 404 returned error can't find the container with id c87d2acd49e0e95a247de1ffa8cd346f921879634b71751fc34321eda2e61d5d
Jul 30 20:33:16.874434 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.874400    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9k2t5" event={"ID":"73305c5b-9360-4c08-af2c-99f3ca6fcded","Type":"ContainerStarted","Data":"9a8ce77e847bcbe831c1eff51d7dc6451f229d2c2090269b49cc71f271c70e67"}
Jul 30 20:33:16.874563 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.874454    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9k2t5" event={"ID":"73305c5b-9360-4c08-af2c-99f3ca6fcded","Type":"ContainerStarted","Data":"c87d2acd49e0e95a247de1ffa8cd346f921879634b71751fc34321eda2e61d5d"}
Jul 30 20:33:16.890074 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:16.890028    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9k2t5" podStartSLOduration=0.890012462 podStartE2EDuration="890.012462ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:33:16.888952526 +0000 UTC m=+475.349911351" watchObservedRunningTime="2026-07-30 20:33:16.890012462 +0000 UTC m=+475.350971277"
Jul 30 20:33:33.061729 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.061684    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"]
Jul 30 20:33:33.065944 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.065922    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.069353 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.069324    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 20:33:33.069787 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.069766    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 20:33:33.069922 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.069866    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kxhrx\""
Jul 30 20:33:33.070161 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.070139    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 20:33:33.070250 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.070141    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 20:33:33.082206 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.082183    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"]
Jul 30 20:33:33.174087 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.174057    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71fe5fe6-416c-4479-8fa0-be872a6157e5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.174203 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.174112    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrqqj\" (UniqueName: \"kubernetes.io/projected/71fe5fe6-416c-4479-8fa0-be872a6157e5-kube-api-access-nrqqj\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.174203 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.174178    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71fe5fe6-416c-4479-8fa0-be872a6157e5-tmp\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.174296 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.174205    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71fe5fe6-416c-4479-8fa0-be872a6157e5-manifests\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.174296 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.174225    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71fe5fe6-416c-4479-8fa0-be872a6157e5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.275168 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.275139    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71fe5fe6-416c-4479-8fa0-be872a6157e5-manifests\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.275168 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.275169    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71fe5fe6-416c-4479-8fa0-be872a6157e5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.275302 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.275198    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71fe5fe6-416c-4479-8fa0-be872a6157e5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.275302 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.275247    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrqqj\" (UniqueName: \"kubernetes.io/projected/71fe5fe6-416c-4479-8fa0-be872a6157e5-kube-api-access-nrqqj\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.275302 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.275269    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71fe5fe6-416c-4479-8fa0-be872a6157e5-tmp\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.275539 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.275520    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71fe5fe6-416c-4479-8fa0-be872a6157e5-manifests\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.277719 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.277677    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71fe5fe6-416c-4479-8fa0-be872a6157e5-tmp\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.277954 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.277937    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71fe5fe6-416c-4479-8fa0-be872a6157e5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.277999 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.277965    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71fe5fe6-416c-4479-8fa0-be872a6157e5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.286355 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.286329    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrqqj\" (UniqueName: \"kubernetes.io/projected/71fe5fe6-416c-4479-8fa0-be872a6157e5-kube-api-access-nrqqj\") pod \"opendatahub-operator-controller-manager-7fd799f58b-skt94\" (UID: \"71fe5fe6-416c-4479-8fa0-be872a6157e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.375754 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.375734    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:33.509443 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.509420    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"]
Jul 30 20:33:33.511603 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:33:33.511578    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fe5fe6_416c_4479_8fa0_be872a6157e5.slice/crio-9b3a9a4d983aca1c6086e6de882e0c178b606dc4a5d399884afc62fc180c62c2 WatchSource:0}: Error finding container 9b3a9a4d983aca1c6086e6de882e0c178b606dc4a5d399884afc62fc180c62c2: Status 404 returned error can't find the container with id 9b3a9a4d983aca1c6086e6de882e0c178b606dc4a5d399884afc62fc180c62c2
Jul 30 20:33:33.941359 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:33.941324    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94" event={"ID":"71fe5fe6-416c-4479-8fa0-be872a6157e5","Type":"ContainerStarted","Data":"9b3a9a4d983aca1c6086e6de882e0c178b606dc4a5d399884afc62fc180c62c2"}
Jul 30 20:33:36.958018 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:36.957983    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94" event={"ID":"71fe5fe6-416c-4479-8fa0-be872a6157e5","Type":"ContainerStarted","Data":"5cc53942921064857244fec7f6f916b17439fdc023e5c480e594761c153b1270"}
Jul 30 20:33:37.964064 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:37.963981    2547 generic.go:356] "Generic (PLEG): container finished" podID="71fe5fe6-416c-4479-8fa0-be872a6157e5" containerID="5cc53942921064857244fec7f6f916b17439fdc023e5c480e594761c153b1270" exitCode=0
Jul 30 20:33:37.964396 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:37.964077    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94" event={"ID":"71fe5fe6-416c-4479-8fa0-be872a6157e5","Type":"ContainerDied","Data":"5cc53942921064857244fec7f6f916b17439fdc023e5c480e594761c153b1270"}
Jul 30 20:33:38.972074 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:38.972033    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94" event={"ID":"71fe5fe6-416c-4479-8fa0-be872a6157e5","Type":"ContainerStarted","Data":"263e9e471439c5b63ec5edb4a840312695c4bf270adc2eb1e6dfe29c9e384f6d"}
Jul 30 20:33:38.972505 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:38.972145    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:38.995017 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:38.994963    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94" podStartSLOduration=1.446216538 podStartE2EDuration="5.994947569s" totalImagesPullingTime="3.454001764s" totalInitContainerRuntime="1.094729267s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:33:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:33:38.99312016 +0000 UTC m=+497.454078998" watchObservedRunningTime="2026-07-30 20:33:38.994947569 +0000 UTC m=+497.455906362"
Jul 30 20:33:49.979509 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:49.979471    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fd799f58b-skt94"
Jul 30 20:33:52.341847 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.341818    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"]
Jul 30 20:33:52.348813 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.348796    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.351627 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.351607    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 20:33:52.351909 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.351892    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 20:33:52.369548 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.369530    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"]
Jul 30 20:33:52.412968 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.412949    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1996729-7f1a-45fe-8544-fc47129da3c2-tmp\") pod \"kube-auth-proxy-66684cbb98-8mqzf\" (UID: \"a1996729-7f1a-45fe-8544-fc47129da3c2\") " pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.413058 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.412990    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcv5z\" (UniqueName: \"kubernetes.io/projected/a1996729-7f1a-45fe-8544-fc47129da3c2-kube-api-access-dcv5z\") pod \"kube-auth-proxy-66684cbb98-8mqzf\" (UID: \"a1996729-7f1a-45fe-8544-fc47129da3c2\") " pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.413058 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.413032    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1996729-7f1a-45fe-8544-fc47129da3c2-tls-certs\") pod \"kube-auth-proxy-66684cbb98-8mqzf\" (UID: \"a1996729-7f1a-45fe-8544-fc47129da3c2\") " pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.513961 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.513932    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcv5z\" (UniqueName: \"kubernetes.io/projected/a1996729-7f1a-45fe-8544-fc47129da3c2-kube-api-access-dcv5z\") pod \"kube-auth-proxy-66684cbb98-8mqzf\" (UID: \"a1996729-7f1a-45fe-8544-fc47129da3c2\") " pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.514054 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.513971    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1996729-7f1a-45fe-8544-fc47129da3c2-tls-certs\") pod \"kube-auth-proxy-66684cbb98-8mqzf\" (UID: \"a1996729-7f1a-45fe-8544-fc47129da3c2\") " pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.514054 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.514015    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1996729-7f1a-45fe-8544-fc47129da3c2-tmp\") pod \"kube-auth-proxy-66684cbb98-8mqzf\" (UID: \"a1996729-7f1a-45fe-8544-fc47129da3c2\") " pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.516504 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.516478    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1996729-7f1a-45fe-8544-fc47129da3c2-tmp\") pod \"kube-auth-proxy-66684cbb98-8mqzf\" (UID: \"a1996729-7f1a-45fe-8544-fc47129da3c2\") " pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.516615 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.516596    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1996729-7f1a-45fe-8544-fc47129da3c2-tls-certs\") pod \"kube-auth-proxy-66684cbb98-8mqzf\" (UID: \"a1996729-7f1a-45fe-8544-fc47129da3c2\") " pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.539215 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.539191    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcv5z\" (UniqueName: \"kubernetes.io/projected/a1996729-7f1a-45fe-8544-fc47129da3c2-kube-api-access-dcv5z\") pod \"kube-auth-proxy-66684cbb98-8mqzf\" (UID: \"a1996729-7f1a-45fe-8544-fc47129da3c2\") " pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.658781 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.658718    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"
Jul 30 20:33:52.830345 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:33:52.830321    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1996729_7f1a_45fe_8544_fc47129da3c2.slice/crio-e17140b32dda5fc021a1cac616bfceae9281d0c1320371851114150b741aa940 WatchSource:0}: Error finding container e17140b32dda5fc021a1cac616bfceae9281d0c1320371851114150b741aa940: Status 404 returned error can't find the container with id e17140b32dda5fc021a1cac616bfceae9281d0c1320371851114150b741aa940
Jul 30 20:33:52.831692 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.831671    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf"]
Jul 30 20:33:52.863081 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.863053    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8nzgh"]
Jul 30 20:33:52.866282 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.866266    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:52.872982 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.872945    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 30 20:33:52.873329 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.873188    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dk85q\""
Jul 30 20:33:52.916452 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.916398    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8nzgh"]
Jul 30 20:33:52.917889 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.917852    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mqnh\" (UniqueName: \"kubernetes.io/projected/fd289d15-b395-4a6b-94e2-36500578a89e-kube-api-access-2mqnh\") pod \"odh-model-controller-7998c67dc4-8nzgh\" (UID: \"fd289d15-b395-4a6b-94e2-36500578a89e\") " pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:52.917973 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:52.917938    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd289d15-b395-4a6b-94e2-36500578a89e-cert\") pod \"odh-model-controller-7998c67dc4-8nzgh\" (UID: \"fd289d15-b395-4a6b-94e2-36500578a89e\") " pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:53.019240 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:53.019214    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mqnh\" (UniqueName: \"kubernetes.io/projected/fd289d15-b395-4a6b-94e2-36500578a89e-kube-api-access-2mqnh\") pod \"odh-model-controller-7998c67dc4-8nzgh\" (UID: \"fd289d15-b395-4a6b-94e2-36500578a89e\") " pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:53.019375 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:53.019258    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd289d15-b395-4a6b-94e2-36500578a89e-cert\") pod \"odh-model-controller-7998c67dc4-8nzgh\" (UID: \"fd289d15-b395-4a6b-94e2-36500578a89e\") " pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:53.019375 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:33:53.019353    2547 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 30 20:33:53.019492 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:33:53.019410    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd289d15-b395-4a6b-94e2-36500578a89e-cert podName:fd289d15-b395-4a6b-94e2-36500578a89e nodeName:}" failed. No retries permitted until 2026-07-30 20:33:53.51939372 +0000 UTC m=+511.980352509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fd289d15-b395-4a6b-94e2-36500578a89e-cert") pod "odh-model-controller-7998c67dc4-8nzgh" (UID: "fd289d15-b395-4a6b-94e2-36500578a89e") : secret "odh-model-controller-webhook-cert" not found
Jul 30 20:33:53.038338 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:53.038300    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf" event={"ID":"a1996729-7f1a-45fe-8544-fc47129da3c2","Type":"ContainerStarted","Data":"e17140b32dda5fc021a1cac616bfceae9281d0c1320371851114150b741aa940"}
Jul 30 20:33:53.039411 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:53.039390    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mqnh\" (UniqueName: \"kubernetes.io/projected/fd289d15-b395-4a6b-94e2-36500578a89e-kube-api-access-2mqnh\") pod \"odh-model-controller-7998c67dc4-8nzgh\" (UID: \"fd289d15-b395-4a6b-94e2-36500578a89e\") " pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:53.523635 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:53.523605    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd289d15-b395-4a6b-94e2-36500578a89e-cert\") pod \"odh-model-controller-7998c67dc4-8nzgh\" (UID: \"fd289d15-b395-4a6b-94e2-36500578a89e\") " pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:53.526046 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:53.526024    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd289d15-b395-4a6b-94e2-36500578a89e-cert\") pod \"odh-model-controller-7998c67dc4-8nzgh\" (UID: \"fd289d15-b395-4a6b-94e2-36500578a89e\") " pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:53.776929 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:53.776851    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:53.971221 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:53.971184    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8nzgh"]
Jul 30 20:33:53.974457 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:33:53.974376    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd289d15_b395_4a6b_94e2_36500578a89e.slice/crio-cf2f81bb4309f97db20137c3a5081f0cb56de392b9f37459b80b621f76e3b9c8 WatchSource:0}: Error finding container cf2f81bb4309f97db20137c3a5081f0cb56de392b9f37459b80b621f76e3b9c8: Status 404 returned error can't find the container with id cf2f81bb4309f97db20137c3a5081f0cb56de392b9f37459b80b621f76e3b9c8
Jul 30 20:33:53.996900 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:53.996869    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dldl8"]
Jul 30 20:33:54.000675 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.000652    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:33:54.004179 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.004161    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 30 20:33:54.025936 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.025880    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dldl8"]
Jul 30 20:33:54.044664 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.044606    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh" event={"ID":"fd289d15-b395-4a6b-94e2-36500578a89e","Type":"ContainerStarted","Data":"cf2f81bb4309f97db20137c3a5081f0cb56de392b9f37459b80b621f76e3b9c8"}
Jul 30 20:33:54.130365 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.130336    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-656hv\" (UniqueName: \"kubernetes.io/projected/63b4f0a2-c8a6-40a6-b00f-cd981d354b26-kube-api-access-656hv\") pod \"model-serving-api-54d89bd5fc-dldl8\" (UID: \"63b4f0a2-c8a6-40a6-b00f-cd981d354b26\") " pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:33:54.130512 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.130404    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b4f0a2-c8a6-40a6-b00f-cd981d354b26-tls-certs\") pod \"model-serving-api-54d89bd5fc-dldl8\" (UID: \"63b4f0a2-c8a6-40a6-b00f-cd981d354b26\") " pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:33:54.232012 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.231982    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-656hv\" (UniqueName: \"kubernetes.io/projected/63b4f0a2-c8a6-40a6-b00f-cd981d354b26-kube-api-access-656hv\") pod \"model-serving-api-54d89bd5fc-dldl8\" (UID: \"63b4f0a2-c8a6-40a6-b00f-cd981d354b26\") " pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:33:54.232177 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.232066    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b4f0a2-c8a6-40a6-b00f-cd981d354b26-tls-certs\") pod \"model-serving-api-54d89bd5fc-dldl8\" (UID: \"63b4f0a2-c8a6-40a6-b00f-cd981d354b26\") " pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:33:54.232177 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:33:54.232164    2547 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 30 20:33:54.232304 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:33:54.232241    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63b4f0a2-c8a6-40a6-b00f-cd981d354b26-tls-certs podName:63b4f0a2-c8a6-40a6-b00f-cd981d354b26 nodeName:}" failed. No retries permitted until 2026-07-30 20:33:54.73221936 +0000 UTC m=+513.193178150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/63b4f0a2-c8a6-40a6-b00f-cd981d354b26-tls-certs") pod "model-serving-api-54d89bd5fc-dldl8" (UID: "63b4f0a2-c8a6-40a6-b00f-cd981d354b26") : secret "model-serving-api-tls" not found
Jul 30 20:33:54.247098 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.247046    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-656hv\" (UniqueName: \"kubernetes.io/projected/63b4f0a2-c8a6-40a6-b00f-cd981d354b26-kube-api-access-656hv\") pod \"model-serving-api-54d89bd5fc-dldl8\" (UID: \"63b4f0a2-c8a6-40a6-b00f-cd981d354b26\") " pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:33:54.302418 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.302337    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"]
Jul 30 20:33:54.304971 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.304946    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.307850 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.307823    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 20:33:54.308283 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.308141    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 30 20:33:54.308283 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.308149    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cjcjr\""
Jul 30 20:33:54.308283 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.308183    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 30 20:33:54.308283 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.308185    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 30 20:33:54.316859 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.316822    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"]
Jul 30 20:33:54.433999 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.433965    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqzjc\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-kube-api-access-jqzjc\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.434329 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.434020    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e71c6f8-4463-42dd-b381-e1e45f8606fd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.434329 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.434208    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.434329 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.434250    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.434329 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.434281    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.434329 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.434313    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.434665 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.434347    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.535233 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.535196    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.535644 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.535257    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.535644 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.535288    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.535644 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.535305    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.535644 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.535349    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqzjc\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-kube-api-access-jqzjc\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.535644 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.535392    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e71c6f8-4463-42dd-b381-e1e45f8606fd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.535644 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.535495    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.536628 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.536131    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.538048 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.538000    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.538280 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.538258    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e71c6f8-4463-42dd-b381-e1e45f8606fd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.538552 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.538534    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.538689 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.538646    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.544676 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.544652    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqzjc\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-kube-api-access-jqzjc\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.544864 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.544841    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-ldzz2\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.617940 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.617913    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:33:54.738892 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.738478    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b4f0a2-c8a6-40a6-b00f-cd981d354b26-tls-certs\") pod \"model-serving-api-54d89bd5fc-dldl8\" (UID: \"63b4f0a2-c8a6-40a6-b00f-cd981d354b26\") " pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:33:54.742179 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.742155    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b4f0a2-c8a6-40a6-b00f-cd981d354b26-tls-certs\") pod \"model-serving-api-54d89bd5fc-dldl8\" (UID: \"63b4f0a2-c8a6-40a6-b00f-cd981d354b26\") " pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:33:54.797090 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.797007    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"]
Jul 30 20:33:54.913891 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:54.913562    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:33:55.561862 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:33:55.561814    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e71c6f8_4463_42dd_b381_e1e45f8606fd.slice/crio-d947c94d62455a7d21fa8e9febb1c99c29a22b1f1ccb2345b430669da1e86ad2 WatchSource:0}: Error finding container d947c94d62455a7d21fa8e9febb1c99c29a22b1f1ccb2345b430669da1e86ad2: Status 404 returned error can't find the container with id d947c94d62455a7d21fa8e9febb1c99c29a22b1f1ccb2345b430669da1e86ad2
Jul 30 20:33:55.719176 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:55.719146    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dldl8"]
Jul 30 20:33:56.056846 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:56.056815    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2" event={"ID":"6e71c6f8-4463-42dd-b381-e1e45f8606fd","Type":"ContainerStarted","Data":"d947c94d62455a7d21fa8e9febb1c99c29a22b1f1ccb2345b430669da1e86ad2"}
Jul 30 20:33:56.058773 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:56.058742    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf" event={"ID":"a1996729-7f1a-45fe-8544-fc47129da3c2","Type":"ContainerStarted","Data":"486edf826f7555e946b09c443fc1f3b26f3a9d6da643acc72f4fcf24813786c3"}
Jul 30 20:33:56.078766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:56.078723    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66684cbb98-8mqzf" podStartSLOduration=1.305855492 podStartE2EDuration="4.078683202s" totalImagesPullingTime="2.77282771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:33:56.077695649 +0000 UTC m=+514.538654473" watchObservedRunningTime="2026-07-30 20:33:56.078683202 +0000 UTC m=+514.539642015"
Jul 30 20:33:56.921735 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:33:56.921677    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b4f0a2_c8a6_40a6_b00f_cd981d354b26.slice/crio-f12df8f561f16d92db685322f85c8cf7126774932f4e653c284953155c7d673e WatchSource:0}: Error finding container f12df8f561f16d92db685322f85c8cf7126774932f4e653c284953155c7d673e: Status 404 returned error can't find the container with id f12df8f561f16d92db685322f85c8cf7126774932f4e653c284953155c7d673e
Jul 30 20:33:57.064228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:57.064197    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh" event={"ID":"fd289d15-b395-4a6b-94e2-36500578a89e","Type":"ContainerStarted","Data":"d606917d4423b07985d2d8dbd9bed4ba8fddf77893c86c84969b72b054f38844"}
Jul 30 20:33:57.064343 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:57.064281    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:33:57.065466 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:57.065442    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dldl8" event={"ID":"63b4f0a2-c8a6-40a6-b00f-cd981d354b26","Type":"ContainerStarted","Data":"f12df8f561f16d92db685322f85c8cf7126774932f4e653c284953155c7d673e"}
Jul 30 20:33:57.082176 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:57.082137    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh" podStartSLOduration=2.085827616 podStartE2EDuration="5.082124457s" totalImagesPullingTime="2.996296841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:33:57.079598125 +0000 UTC m=+515.540556938" watchObservedRunningTime="2026-07-30 20:33:57.082124457 +0000 UTC m=+515.543083269"
Jul 30 20:33:59.823017 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.822879    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"]
Jul 30 20:33:59.830736 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.830684    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:33:59.839450 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.839395    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"]
Jul 30 20:33:59.981514 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.981479    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b944a48c-5904-4e31-b715-765956c18303-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:33:59.981693 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.981543    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b944a48c-5904-4e31-b715-765956c18303-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:33:59.981693 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.981629    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b944a48c-5904-4e31-b715-765956c18303-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:33:59.981693 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.981665    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqqkr\" (UniqueName: \"kubernetes.io/projected/b944a48c-5904-4e31-b715-765956c18303-kube-api-access-mqqkr\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:33:59.981871 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.981694    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b944a48c-5904-4e31-b715-765956c18303-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:33:59.981871 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.981805    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b944a48c-5904-4e31-b715-765956c18303-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:33:59.981871 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:33:59.981839    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b944a48c-5904-4e31-b715-765956c18303-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.082347 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.082258    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b944a48c-5904-4e31-b715-765956c18303-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.082347 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.082302    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqqkr\" (UniqueName: \"kubernetes.io/projected/b944a48c-5904-4e31-b715-765956c18303-kube-api-access-mqqkr\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.082347 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.082334    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b944a48c-5904-4e31-b715-765956c18303-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.082620 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.082405    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b944a48c-5904-4e31-b715-765956c18303-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.082620 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.082439    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b944a48c-5904-4e31-b715-765956c18303-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.082620 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.082486    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b944a48c-5904-4e31-b715-765956c18303-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.082620 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.082527    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b944a48c-5904-4e31-b715-765956c18303-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.083141 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.083087    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b944a48c-5904-4e31-b715-765956c18303-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.083969 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.083935    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dldl8" event={"ID":"63b4f0a2-c8a6-40a6-b00f-cd981d354b26","Type":"ContainerStarted","Data":"ec82ea00066b4b9dc8339025865f5262c89c5566c6a4babe545d38a19116290e"}
Jul 30 20:34:00.084646 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.084624    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:34:00.085065 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.085037    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b944a48c-5904-4e31-b715-765956c18303-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.085410 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.085387    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b944a48c-5904-4e31-b715-765956c18303-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.085517 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.085449    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b944a48c-5904-4e31-b715-765956c18303-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.085815 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.085784    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b944a48c-5904-4e31-b715-765956c18303-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.099727 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.099685    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b944a48c-5904-4e31-b715-765956c18303-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.100333 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.100311    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqqkr\" (UniqueName: \"kubernetes.io/projected/b944a48c-5904-4e31-b715-765956c18303-kube-api-access-mqqkr\") pod \"istiod-openshift-gateway-5677ccb49b-ccnsn\" (UID: \"b944a48c-5904-4e31-b715-765956c18303\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.106803 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.106764    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dldl8" podStartSLOduration=4.919229196 podStartE2EDuration="7.106751633s" totalImagesPullingTime="2.187522437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:33:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:34:00.105416184 +0000 UTC m=+518.566374998" watchObservedRunningTime="2026-07-30 20:34:00.106751633 +0000 UTC m=+518.567710444"
Jul 30 20:34:00.145350 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.145333    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:00.204249 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.204072    2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 30 20:34:00.204249 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.204139    2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 30 20:34:00.286319 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.286293    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"]
Jul 30 20:34:00.289564 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:34:00.289516    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb944a48c_5904_4e31_b715_765956c18303.slice/crio-8e4bc64ce16ed26c5e4a09be1524310c89c3692e782560ee38f41e04eff4564b WatchSource:0}: Error finding container 8e4bc64ce16ed26c5e4a09be1524310c89c3692e782560ee38f41e04eff4564b: Status 404 returned error can't find the container with id 8e4bc64ce16ed26c5e4a09be1524310c89c3692e782560ee38f41e04eff4564b
Jul 30 20:34:00.293252 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.293222    2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 30 20:34:00.293357 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:00.293293    2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 30 20:34:01.090851 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.090806    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn" event={"ID":"b944a48c-5904-4e31-b715-765956c18303","Type":"ContainerStarted","Data":"d20da3b9d4aaa138b0c9a06d95059ce9bff9dfe49bf2ed9cfb95503c103cd4ce"}
Jul 30 20:34:01.091326 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.090908    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn" event={"ID":"b944a48c-5904-4e31-b715-765956c18303","Type":"ContainerStarted","Data":"8e4bc64ce16ed26c5e4a09be1524310c89c3692e782560ee38f41e04eff4564b"}
Jul 30 20:34:01.091326 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.090937    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:01.092665 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.092634    2547 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-ccnsn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 20:34:01.092836 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.092684    2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn" podUID="b944a48c-5904-4e31-b715-765956c18303" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 20:34:01.092836 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.092768    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2" event={"ID":"6e71c6f8-4463-42dd-b381-e1e45f8606fd","Type":"ContainerStarted","Data":"3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8"}
Jul 30 20:34:01.092836 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.092817    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:34:01.094120 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.094093    2547 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-ldzz2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 20:34:01.094211 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.094139    2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2" podUID="6e71c6f8-4463-42dd-b381-e1e45f8606fd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 20:34:01.117231 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.117194    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn" podStartSLOduration=2.117181684 podStartE2EDuration="2.117181684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:34:01.114996965 +0000 UTC m=+519.575955791" watchObservedRunningTime="2026-07-30 20:34:01.117181684 +0000 UTC m=+519.578140477"
Jul 30 20:34:01.136603 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:01.136564    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2" podStartSLOduration=2.4989058699999998 podStartE2EDuration="7.136552958s" totalImagesPullingTime="4.637647088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:34:01.13503874 +0000 UTC m=+519.595997561" watchObservedRunningTime="2026-07-30 20:34:01.136552958 +0000 UTC m=+519.597511769"
Jul 30 20:34:02.096852 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:02.096824    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:34:02.097188 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:02.096883    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ccnsn"
Jul 30 20:34:02.186040 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:02.186014    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"]
Jul 30 20:34:04.105028 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.104963    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2" podUID="6e71c6f8-4463-42dd-b381-e1e45f8606fd" containerName="discovery" containerID="cri-o://3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8" gracePeriod=30
Jul 30 20:34:04.347919 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.347897    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:34:04.421013 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.420948    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-kubeconfig\") pod \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") "
Jul 30 20:34:04.421013 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.420982    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-ca-configmap\") pod \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") "
Jul 30 20:34:04.421175 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.421021    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-dns-cert\") pod \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") "
Jul 30 20:34:04.421175 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.421050    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqzjc\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-kube-api-access-jqzjc\") pod \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") "
Jul 30 20:34:04.421175 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.421089    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-token\") pod \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") "
Jul 30 20:34:04.421175 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.421126    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e71c6f8-4463-42dd-b381-e1e45f8606fd-local-certs\") pod \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") "
Jul 30 20:34:04.421175 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.421148    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-cacerts\") pod \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\" (UID: \"6e71c6f8-4463-42dd-b381-e1e45f8606fd\") "
Jul 30 20:34:04.421414 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.421375    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6e71c6f8-4463-42dd-b381-e1e45f8606fd" (UID: "6e71c6f8-4463-42dd-b381-e1e45f8606fd"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:34:04.423607 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.423576    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e71c6f8-4463-42dd-b381-e1e45f8606fd-local-certs" (OuterVolumeSpecName: "local-certs") pod "6e71c6f8-4463-42dd-b381-e1e45f8606fd" (UID: "6e71c6f8-4463-42dd-b381-e1e45f8606fd"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:34:04.423607 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.423600    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6e71c6f8-4463-42dd-b381-e1e45f8606fd" (UID: "6e71c6f8-4463-42dd-b381-e1e45f8606fd"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:34:04.423799 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.423670    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-kube-api-access-jqzjc" (OuterVolumeSpecName: "kube-api-access-jqzjc") pod "6e71c6f8-4463-42dd-b381-e1e45f8606fd" (UID: "6e71c6f8-4463-42dd-b381-e1e45f8606fd"). InnerVolumeSpecName "kube-api-access-jqzjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:34:04.423799 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.423668    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e71c6f8-4463-42dd-b381-e1e45f8606fd" (UID: "6e71c6f8-4463-42dd-b381-e1e45f8606fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:34:04.423921 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.423882    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-cacerts" (OuterVolumeSpecName: "cacerts") pod "6e71c6f8-4463-42dd-b381-e1e45f8606fd" (UID: "6e71c6f8-4463-42dd-b381-e1e45f8606fd"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:34:04.424036 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.424013    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6e71c6f8-4463-42dd-b381-e1e45f8606fd" (UID: "6e71c6f8-4463-42dd-b381-e1e45f8606fd"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:34:04.522278 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.522260    2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-token\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:34:04.522278 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.522278    2547 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e71c6f8-4463-42dd-b381-e1e45f8606fd-local-certs\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:34:04.522417 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.522287    2547 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-cacerts\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:34:04.522417 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.522294    2547 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-kubeconfig\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:34:04.522417 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.522302    2547 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-ca-configmap\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:34:04.522417 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.522311    2547 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e71c6f8-4463-42dd-b381-e1e45f8606fd-istio-csr-dns-cert\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:34:04.522417 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:04.522319    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqzjc\" (UniqueName: \"kubernetes.io/projected/6e71c6f8-4463-42dd-b381-e1e45f8606fd-kube-api-access-jqzjc\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:34:05.111107 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:05.111071    2547 generic.go:356] "Generic (PLEG): container finished" podID="6e71c6f8-4463-42dd-b381-e1e45f8606fd" containerID="3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8" exitCode=0
Jul 30 20:34:05.111481 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:05.111142    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"
Jul 30 20:34:05.111481 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:05.111152    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2" event={"ID":"6e71c6f8-4463-42dd-b381-e1e45f8606fd","Type":"ContainerDied","Data":"3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8"}
Jul 30 20:34:05.111481 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:05.111191    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2" event={"ID":"6e71c6f8-4463-42dd-b381-e1e45f8606fd","Type":"ContainerDied","Data":"d947c94d62455a7d21fa8e9febb1c99c29a22b1f1ccb2345b430669da1e86ad2"}
Jul 30 20:34:05.111481 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:05.111212    2547 scope.go:117] "RemoveContainer" containerID="3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8"
Jul 30 20:34:05.121269 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:05.121250    2547 scope.go:117] "RemoveContainer" containerID="3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8"
Jul 30 20:34:05.121577 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:34:05.121546    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8\": container with ID starting with 3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8 not found: ID does not exist" containerID="3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8"
Jul 30 20:34:05.121685 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:05.121578    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8"} err="failed to get container status \"3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8\": rpc error: code = NotFound desc = could not find container \"3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8\": container with ID starting with 3960addadaf1937e8f87f0ae0fa2f0bd57d39dc09b1b0f8d3c0c6f18b77ca7b8 not found: ID does not exist"
Jul 30 20:34:05.140471 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:05.140445    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"]
Jul 30 20:34:05.145627 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:05.145606    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ldzz2"]
Jul 30 20:34:06.072215 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:06.072182    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e71c6f8-4463-42dd-b381-e1e45f8606fd" path="/var/lib/kubelet/pods/6e71c6f8-4463-42dd-b381-e1e45f8606fd/volumes"
Jul 30 20:34:08.073333 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:08.073309    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-8nzgh"
Jul 30 20:34:12.100627 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:34:12.100599    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dldl8"
Jul 30 20:35:13.890294 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.890259    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6554b88db9-7bgfv"]
Jul 30 20:35:13.891036 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.890874    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e71c6f8-4463-42dd-b381-e1e45f8606fd" containerName="discovery"
Jul 30 20:35:13.891036 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.890899    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e71c6f8-4463-42dd-b381-e1e45f8606fd" containerName="discovery"
Jul 30 20:35:13.891036 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.890984    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e71c6f8-4463-42dd-b381-e1e45f8606fd" containerName="discovery"
Jul 30 20:35:13.893375 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.893353    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:13.897448 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.897425    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 20:35:13.897570 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.897478    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mf675\""
Jul 30 20:35:13.897570 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.897449    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 20:35:13.897570 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.897477    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 20:35:13.897782 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.897728    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 20:35:13.898080 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.898062    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 20:35:13.902836 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.902803    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 20:35:13.905341 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.905317    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6554b88db9-7bgfv"]
Jul 30 20:35:13.910959 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.910932    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-console-config\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:13.911071 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.910976    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-trusted-ca-bundle\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:13.911071 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.911048    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp4qw\" (UniqueName: \"kubernetes.io/projected/c144dbdf-cb96-4569-a126-ecac59770f24-kube-api-access-mp4qw\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:13.911225 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.911070    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-oauth-serving-cert\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:13.911225 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.911129    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c144dbdf-cb96-4569-a126-ecac59770f24-console-oauth-config\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:13.911225 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.911170    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-service-ca\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:13.911408 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:13.911225    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c144dbdf-cb96-4569-a126-ecac59770f24-console-serving-cert\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.012322 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.012296    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-console-config\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.012419 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.012328    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-trusted-ca-bundle\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.012419 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.012365    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp4qw\" (UniqueName: \"kubernetes.io/projected/c144dbdf-cb96-4569-a126-ecac59770f24-kube-api-access-mp4qw\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.012419 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.012386    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-oauth-serving-cert\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.012419 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.012411    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c144dbdf-cb96-4569-a126-ecac59770f24-console-oauth-config\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.012598 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.012431    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-service-ca\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.012598 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.012479    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c144dbdf-cb96-4569-a126-ecac59770f24-console-serving-cert\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.013202 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.013175    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-console-config\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.013324 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.013187    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-service-ca\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.013384 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.013241    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-trusted-ca-bundle\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.013384 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.013372    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c144dbdf-cb96-4569-a126-ecac59770f24-oauth-serving-cert\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.014955 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.014933    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c144dbdf-cb96-4569-a126-ecac59770f24-console-oauth-config\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.015057 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.015040    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c144dbdf-cb96-4569-a126-ecac59770f24-console-serving-cert\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.021886 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.021865    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp4qw\" (UniqueName: \"kubernetes.io/projected/c144dbdf-cb96-4569-a126-ecac59770f24-kube-api-access-mp4qw\") pod \"console-6554b88db9-7bgfv\" (UID: \"c144dbdf-cb96-4569-a126-ecac59770f24\") " pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.205012 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.204954    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:14.351590 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.351534    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6554b88db9-7bgfv"]
Jul 30 20:35:14.353965 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:35:14.353929    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc144dbdf_cb96_4569_a126_ecac59770f24.slice/crio-16029a5570a6e873575eec99b357e5edacf79522d67413f8163291dbf4d12fc4 WatchSource:0}: Error finding container 16029a5570a6e873575eec99b357e5edacf79522d67413f8163291dbf4d12fc4: Status 404 returned error can't find the container with id 16029a5570a6e873575eec99b357e5edacf79522d67413f8163291dbf4d12fc4
Jul 30 20:35:14.430364 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.430330    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6554b88db9-7bgfv" event={"ID":"c144dbdf-cb96-4569-a126-ecac59770f24","Type":"ContainerStarted","Data":"9feb81cf956d0911f485ce8e9040114335c0ac28d220578f9230c6514b3d8dd3"}
Jul 30 20:35:14.430481 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.430400    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6554b88db9-7bgfv" event={"ID":"c144dbdf-cb96-4569-a126-ecac59770f24","Type":"ContainerStarted","Data":"16029a5570a6e873575eec99b357e5edacf79522d67413f8163291dbf4d12fc4"}
Jul 30 20:35:14.450722 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.450645    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6554b88db9-7bgfv" podStartSLOduration=1.450625338 podStartE2EDuration="1.450625338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:35:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:35:14.449986121 +0000 UTC m=+592.910944935" watchObservedRunningTime="2026-07-30 20:35:14.450625338 +0000 UTC m=+592.911584151"
Jul 30 20:35:14.529385 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.529316    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"]
Jul 30 20:35:14.531606 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.531588    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"
Jul 30 20:35:14.534815 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.534798    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-txq6j\""
Jul 30 20:35:14.535078 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.535061    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 20:35:14.535321 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.535304    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 30 20:35:14.535403 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.535322    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 20:35:14.543677 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.543655    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"]
Jul 30 20:35:14.618321 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.618297    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvwn4\" (UniqueName: \"kubernetes.io/projected/6b5ea85f-7ef7-445e-89af-0c2febe43f59-kube-api-access-jvwn4\") pod \"dns-operator-controller-manager-5985564776-ftk76\" (UID: \"6b5ea85f-7ef7-445e-89af-0c2febe43f59\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"
Jul 30 20:35:14.719180 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.719154    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvwn4\" (UniqueName: \"kubernetes.io/projected/6b5ea85f-7ef7-445e-89af-0c2febe43f59-kube-api-access-jvwn4\") pod \"dns-operator-controller-manager-5985564776-ftk76\" (UID: \"6b5ea85f-7ef7-445e-89af-0c2febe43f59\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"
Jul 30 20:35:14.728088 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.728068    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvwn4\" (UniqueName: \"kubernetes.io/projected/6b5ea85f-7ef7-445e-89af-0c2febe43f59-kube-api-access-jvwn4\") pod \"dns-operator-controller-manager-5985564776-ftk76\" (UID: \"6b5ea85f-7ef7-445e-89af-0c2febe43f59\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"
Jul 30 20:35:14.842506 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.842439    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"
Jul 30 20:35:14.992936 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:14.992912    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"]
Jul 30 20:35:14.995372 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:35:14.995326    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5ea85f_7ef7_445e_89af_0c2febe43f59.slice/crio-3f0c2c5fc7a160f8ed4b10c0b2e1a195852c596efe4dc6c97e8a4c6df0b15d45 WatchSource:0}: Error finding container 3f0c2c5fc7a160f8ed4b10c0b2e1a195852c596efe4dc6c97e8a4c6df0b15d45: Status 404 returned error can't find the container with id 3f0c2c5fc7a160f8ed4b10c0b2e1a195852c596efe4dc6c97e8a4c6df0b15d45
Jul 30 20:35:15.436488 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:15.436448    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76" event={"ID":"6b5ea85f-7ef7-445e-89af-0c2febe43f59","Type":"ContainerStarted","Data":"3f0c2c5fc7a160f8ed4b10c0b2e1a195852c596efe4dc6c97e8a4c6df0b15d45"}
Jul 30 20:35:17.468342 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:17.468306    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-llkvp"]
Jul 30 20:35:17.470865 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:17.470839    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-llkvp"
Jul 30 20:35:17.474409 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:17.474386    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7xr9x\""
Jul 30 20:35:17.495396 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:17.495374    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-llkvp"]
Jul 30 20:35:17.546330 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:17.546297    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h98pc\" (UniqueName: \"kubernetes.io/projected/782e133f-136b-456b-af50-3724115aeda7-kube-api-access-h98pc\") pod \"authorino-operator-68d88948d8-llkvp\" (UID: \"782e133f-136b-456b-af50-3724115aeda7\") " pod="kuadrant-system/authorino-operator-68d88948d8-llkvp"
Jul 30 20:35:17.647847 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:17.647813    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h98pc\" (UniqueName: \"kubernetes.io/projected/782e133f-136b-456b-af50-3724115aeda7-kube-api-access-h98pc\") pod \"authorino-operator-68d88948d8-llkvp\" (UID: \"782e133f-136b-456b-af50-3724115aeda7\") " pod="kuadrant-system/authorino-operator-68d88948d8-llkvp"
Jul 30 20:35:17.660588 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:17.660556    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h98pc\" (UniqueName: \"kubernetes.io/projected/782e133f-136b-456b-af50-3724115aeda7-kube-api-access-h98pc\") pod \"authorino-operator-68d88948d8-llkvp\" (UID: \"782e133f-136b-456b-af50-3724115aeda7\") " pod="kuadrant-system/authorino-operator-68d88948d8-llkvp"
Jul 30 20:35:17.785818 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:17.785793    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-llkvp"
Jul 30 20:35:17.949515 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:17.949490    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-llkvp"]
Jul 30 20:35:17.952886 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:35:17.952862    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod782e133f_136b_456b_af50_3724115aeda7.slice/crio-ac4f844a6fe016630bba493f568a489d721c5d997b217ae9eeb39d39420437d4 WatchSource:0}: Error finding container ac4f844a6fe016630bba493f568a489d721c5d997b217ae9eeb39d39420437d4: Status 404 returned error can't find the container with id ac4f844a6fe016630bba493f568a489d721c5d997b217ae9eeb39d39420437d4
Jul 30 20:35:18.452730 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:18.452687    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76" event={"ID":"6b5ea85f-7ef7-445e-89af-0c2febe43f59","Type":"ContainerStarted","Data":"8d8cdc28e59cb1bec7ed15f59c1aa3243f537ec010fcb8dd937c77d23195b4e7"}
Jul 30 20:35:18.452906 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:18.452860    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"
Jul 30 20:35:18.454000 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:18.453978    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-llkvp" event={"ID":"782e133f-136b-456b-af50-3724115aeda7","Type":"ContainerStarted","Data":"ac4f844a6fe016630bba493f568a489d721c5d997b217ae9eeb39d39420437d4"}
Jul 30 20:35:18.477659 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:18.477611    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76" podStartSLOduration=1.8064194009999999 podStartE2EDuration="4.477597706s" totalImagesPullingTime="2.671178305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:35:18.477325723 +0000 UTC m=+596.938284536" watchObservedRunningTime="2026-07-30 20:35:18.477597706 +0000 UTC m=+596.938556519"
Jul 30 20:35:21.742152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.742113    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"]
Jul 30 20:35:21.745494 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.745472    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:21.753661 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.753639    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4j9mr\""
Jul 30 20:35:21.763210 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.763182    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"]
Jul 30 20:35:21.778920 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.778895    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/846a2a7a-39f5-4feb-8c0b-f03664c22d44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" (UID: \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:21.779036 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.778936    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcqq8\" (UniqueName: \"kubernetes.io/projected/846a2a7a-39f5-4feb-8c0b-f03664c22d44-kube-api-access-hcqq8\") pod \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" (UID: \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:21.880259 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.880230    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/846a2a7a-39f5-4feb-8c0b-f03664c22d44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" (UID: \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:21.880369 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.880269    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcqq8\" (UniqueName: \"kubernetes.io/projected/846a2a7a-39f5-4feb-8c0b-f03664c22d44-kube-api-access-hcqq8\") pod \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" (UID: \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:21.880569 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.880551    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/846a2a7a-39f5-4feb-8c0b-f03664c22d44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" (UID: \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:21.889870 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:21.889843    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcqq8\" (UniqueName: \"kubernetes.io/projected/846a2a7a-39f5-4feb-8c0b-f03664c22d44-kube-api-access-hcqq8\") pod \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" (UID: \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:22.000127 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:22.000067    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:35:22.001243 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:22.001220    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:35:22.003774 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:22.003753    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:35:22.004739 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:22.004720    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:35:22.059966 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:22.059947    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4j9mr\""
Jul 30 20:35:22.067523 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:22.067504    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:22.213777 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:22.213731    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"]
Jul 30 20:35:22.218949 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:35:22.218918    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod846a2a7a_39f5_4feb_8c0b_f03664c22d44.slice/crio-2b583b9970767cb8a093d5957d0320021ce1a0d48ce56cb3308e2f4bf6de6f58 WatchSource:0}: Error finding container 2b583b9970767cb8a093d5957d0320021ce1a0d48ce56cb3308e2f4bf6de6f58: Status 404 returned error can't find the container with id 2b583b9970767cb8a093d5957d0320021ce1a0d48ce56cb3308e2f4bf6de6f58
Jul 30 20:35:22.473280 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:22.473244    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz" event={"ID":"846a2a7a-39f5-4feb-8c0b-f03664c22d44","Type":"ContainerStarted","Data":"2b583b9970767cb8a093d5957d0320021ce1a0d48ce56cb3308e2f4bf6de6f58"}
Jul 30 20:35:24.205943 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:24.205826    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:24.205943 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:24.205893    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:24.214580 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:24.211533    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:24.490093 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:24.490014    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6554b88db9-7bgfv"
Jul 30 20:35:26.496870 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:26.496795    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz" event={"ID":"846a2a7a-39f5-4feb-8c0b-f03664c22d44","Type":"ContainerStarted","Data":"2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e"}
Jul 30 20:35:26.497206 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:26.496907    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:26.518338 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:26.518230    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz" podStartSLOduration=1.4935516070000001 podStartE2EDuration="5.518213108s" totalImagesPullingTime="4.024661501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:35:26.516754248 +0000 UTC m=+604.977713061" watchObservedRunningTime="2026-07-30 20:35:26.518213108 +0000 UTC m=+604.979171921"
Jul 30 20:35:28.507586 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:28.507551    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-llkvp" event={"ID":"782e133f-136b-456b-af50-3724115aeda7","Type":"ContainerStarted","Data":"b9edfd1ec6ed93efd3f36383495e4580bfdc73355a0b31ec8be72ce807a0ceaf"}
Jul 30 20:35:28.508004 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:28.507677    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-llkvp"
Jul 30 20:35:28.542474 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:28.542434    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-llkvp" podStartSLOduration=1.47849131 podStartE2EDuration="11.542419807s" totalImagesPullingTime="10.063928497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:35:28.541025353 +0000 UTC m=+607.001984166" watchObservedRunningTime="2026-07-30 20:35:28.542419807 +0000 UTC m=+607.003378619"
Jul 30 20:35:29.460772 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:29.460743    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-ftk76"
Jul 30 20:35:37.503962 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:37.503923    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.169641 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.169586    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"]
Jul 30 20:35:39.172667 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.172642    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.186751 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.186721    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"]
Jul 30 20:35:39.264415 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.264392    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"]
Jul 30 20:35:39.264582 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.264544    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" containerName="manager" containerID="cri-o://2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e" gracePeriod=2
Jul 30 20:35:39.284326 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.284300    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"]
Jul 30 20:35:39.290432 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.290395    2547 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"846a2a7a-39f5-4feb-8c0b-f03664c22d44\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-tnhqz\": pods \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.298462 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.298437    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"]
Jul 30 20:35:39.298751 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.298728    2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-dvqrp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk" podUID="72de8560-27ae-43bd-a2e4-342041fbdc47"
Jul 30 20:35:39.318336 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.318313    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvqrp\" (UniqueName: \"kubernetes.io/projected/72de8560-27ae-43bd-a2e4-342041fbdc47-kube-api-access-dvqrp\") pod \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" (UID: \"72de8560-27ae-43bd-a2e4-342041fbdc47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.318467 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.318446    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72de8560-27ae-43bd-a2e4-342041fbdc47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" (UID: \"72de8560-27ae-43bd-a2e4-342041fbdc47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.329379 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.329357    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"]
Jul 30 20:35:39.348871 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.348848    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"]
Jul 30 20:35:39.413686 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.413658    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"]
Jul 30 20:35:39.414084 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.414064    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" containerName="manager"
Jul 30 20:35:39.414084 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.414084    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" containerName="manager"
Jul 30 20:35:39.414276 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.414174    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" containerName="manager"
Jul 30 20:35:39.416729 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.416682    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:35:39.420937 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.420874    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72de8560-27ae-43bd-a2e4-342041fbdc47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" (UID: \"72de8560-27ae-43bd-a2e4-342041fbdc47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.421041 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.420996    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvqrp\" (UniqueName: \"kubernetes.io/projected/72de8560-27ae-43bd-a2e4-342041fbdc47-kube-api-access-dvqrp\") pod \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" (UID: \"72de8560-27ae-43bd-a2e4-342041fbdc47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.421766 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.421249    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72de8560-27ae-43bd-a2e4-342041fbdc47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" (UID: \"72de8560-27ae-43bd-a2e4-342041fbdc47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.422229 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.422197    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.426841 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.426818    2547 projected.go:196] Error preparing data for projected volume kube-api-access-dvqrp for pod kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk: failed to fetch token: pod "kuadrant-operator-controller-manager-6bd956ccf-qrwgk" not found
Jul 30 20:35:39.426930 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.426890    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72de8560-27ae-43bd-a2e4-342041fbdc47-kube-api-access-dvqrp podName:72de8560-27ae-43bd-a2e4-342041fbdc47 nodeName:}" failed. No retries permitted until 2026-07-30 20:35:39.926869655 +0000 UTC m=+618.387828463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dvqrp" (UniqueName: "kubernetes.io/projected/72de8560-27ae-43bd-a2e4-342041fbdc47-kube-api-access-dvqrp") pod "kuadrant-operator-controller-manager-6bd956ccf-qrwgk" (UID: "72de8560-27ae-43bd-a2e4-342041fbdc47") : failed to fetch token: pod "kuadrant-operator-controller-manager-6bd956ccf-qrwgk" not found
Jul 30 20:35:39.445193 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.445169    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"]
Jul 30 20:35:39.513344 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.513319    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-llkvp"
Jul 30 20:35:39.521948 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.521923    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3536195d-3a77-4aed-b715-fb7d0ac7c84b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-n4bqt\" (UID: \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:35:39.522045 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.521971    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgg58\" (UniqueName: \"kubernetes.io/projected/3536195d-3a77-4aed-b715-fb7d0ac7c84b-kube-api-access-vgg58\") pod \"kuadrant-operator-controller-manager-68cf589589-n4bqt\" (UID: \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:35:39.530118 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.530097    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.549947 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.549924    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.552407 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.552384    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.567486 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.567463    2547 generic.go:356] "Generic (PLEG): container finished" podID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" containerID="2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e" exitCode=0
Jul 30 20:35:39.567596 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.567525    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.567646 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.567563    2547 scope.go:117] "RemoveContainer" containerID="2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e"
Jul 30 20:35:39.567719 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.567676    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.570410 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.570314    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.573002 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.572979    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.574652 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.574634    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.575953 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.575924    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="72de8560-27ae-43bd-a2e4-342041fbdc47" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.577148 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.577128    2547 scope.go:117] "RemoveContainer" containerID="2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e"
Jul 30 20:35:39.577443 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.577426    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e\": container with ID starting with 2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e not found: ID does not exist" containerID="2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e"
Jul 30 20:35:39.577509 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.577452    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e"} err="failed to get container status \"2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e\": rpc error: code = NotFound desc = could not find container \"2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e\": container with ID starting with 2bf59ee492cdb03757c1c194f34a16f6a91f5c640e2b85c6ea25a0f070867c0e not found: ID does not exist"
Jul 30 20:35:39.578222 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.578198    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.580545 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.580523    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="72de8560-27ae-43bd-a2e4-342041fbdc47" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:39.622914 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.622889    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcqq8\" (UniqueName: \"kubernetes.io/projected/846a2a7a-39f5-4feb-8c0b-f03664c22d44-kube-api-access-hcqq8\") pod \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\" (UID: \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\") "
Jul 30 20:35:39.623004 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.622923    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/846a2a7a-39f5-4feb-8c0b-f03664c22d44-extensions-socket-volume\") pod \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\" (UID: \"846a2a7a-39f5-4feb-8c0b-f03664c22d44\") "
Jul 30 20:35:39.623105 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.623091    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3536195d-3a77-4aed-b715-fb7d0ac7c84b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-n4bqt\" (UID: \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:35:39.623170 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.623130    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgg58\" (UniqueName: \"kubernetes.io/projected/3536195d-3a77-4aed-b715-fb7d0ac7c84b-kube-api-access-vgg58\") pod \"kuadrant-operator-controller-manager-68cf589589-n4bqt\" (UID: \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:35:39.623421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.623396    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3536195d-3a77-4aed-b715-fb7d0ac7c84b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-n4bqt\" (UID: \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:35:39.623421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.623420    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/846a2a7a-39f5-4feb-8c0b-f03664c22d44-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "846a2a7a-39f5-4feb-8c0b-f03664c22d44" (UID: "846a2a7a-39f5-4feb-8c0b-f03664c22d44"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:35:39.625090 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.625070    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/846a2a7a-39f5-4feb-8c0b-f03664c22d44-kube-api-access-hcqq8" (OuterVolumeSpecName: "kube-api-access-hcqq8") pod "846a2a7a-39f5-4feb-8c0b-f03664c22d44" (UID: "846a2a7a-39f5-4feb-8c0b-f03664c22d44"). InnerVolumeSpecName "kube-api-access-hcqq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:35:39.632144 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.632126    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgg58\" (UniqueName: \"kubernetes.io/projected/3536195d-3a77-4aed-b715-fb7d0ac7c84b-kube-api-access-vgg58\") pod \"kuadrant-operator-controller-manager-68cf589589-n4bqt\" (UID: \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:35:39.724176 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.724124    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72de8560-27ae-43bd-a2e4-342041fbdc47-extensions-socket-volume\") pod \"72de8560-27ae-43bd-a2e4-342041fbdc47\" (UID: \"72de8560-27ae-43bd-a2e4-342041fbdc47\") "
Jul 30 20:35:39.724351 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.724331    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72de8560-27ae-43bd-a2e4-342041fbdc47-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "72de8560-27ae-43bd-a2e4-342041fbdc47" (UID: "72de8560-27ae-43bd-a2e4-342041fbdc47"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:35:39.724418 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.724356    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcqq8\" (UniqueName: \"kubernetes.io/projected/846a2a7a-39f5-4feb-8c0b-f03664c22d44-kube-api-access-hcqq8\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:35:39.724418 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.724377    2547 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/846a2a7a-39f5-4feb-8c0b-f03664c22d44-extensions-socket-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:35:39.724418 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.724391    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvqrp\" (UniqueName: \"kubernetes.io/projected/72de8560-27ae-43bd-a2e4-342041fbdc47-kube-api-access-dvqrp\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:35:39.729197 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.729182    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:35:39.825264 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.825240    2547 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72de8560-27ae-43bd-a2e4-342041fbdc47-extensions-socket-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:35:39.869230 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:39.869208    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"]
Jul 30 20:35:39.872259 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:35:39.872230    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3536195d_3a77_4aed_b715_fb7d0ac7c84b.slice/crio-b349fc072cabd492dd3d3b1da2be9336c15e87c8d5959dd19e3805c89e9d1575 WatchSource:0}: Error finding container b349fc072cabd492dd3d3b1da2be9336c15e87c8d5959dd19e3805c89e9d1575: Status 404 returned error can't find the container with id b349fc072cabd492dd3d3b1da2be9336c15e87c8d5959dd19e3805c89e9d1575
Jul 30 20:35:39.880267 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.880240    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-tnhqz\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-tnhqz"
Jul 30 20:35:39.882786 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:39.882761    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="72de8560-27ae-43bd-a2e4-342041fbdc47" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:40.072568 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:40.072540    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72de8560-27ae-43bd-a2e4-342041fbdc47" path="/var/lib/kubelet/pods/72de8560-27ae-43bd-a2e4-342041fbdc47/volumes"
Jul 30 20:35:40.072831 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:40.072818    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="846a2a7a-39f5-4feb-8c0b-f03664c22d44" path="/var/lib/kubelet/pods/846a2a7a-39f5-4feb-8c0b-f03664c22d44/volumes"
Jul 30 20:35:40.574770 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:40.574735    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt" event={"ID":"3536195d-3a77-4aed-b715-fb7d0ac7c84b","Type":"ContainerStarted","Data":"2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d"}
Jul 30 20:35:40.574770 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:40.574769    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:40.575308 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:40.574788    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt" event={"ID":"3536195d-3a77-4aed-b715-fb7d0ac7c84b","Type":"ContainerStarted","Data":"b349fc072cabd492dd3d3b1da2be9336c15e87c8d5959dd19e3805c89e9d1575"}
Jul 30 20:35:40.575308 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:40.574827    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:35:40.600525 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:40.600473    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt" podStartSLOduration=1.600458133 podStartE2EDuration="1.600458133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:35:40.597860152 +0000 UTC m=+619.058818964" watchObservedRunningTime="2026-07-30 20:35:40.600458133 +0000 UTC m=+619.061416944"
Jul 30 20:35:40.600865 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:40.600837    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="72de8560-27ae-43bd-a2e4-342041fbdc47" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:42.070443 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:35:42.070408    2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-qrwgk\" is forbidden: User \"system:node:ip-10-0-134-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-148.ec2.internal' and this object" podUID="72de8560-27ae-43bd-a2e4-342041fbdc47" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-qrwgk"
Jul 30 20:35:51.583929 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:35:51.583822    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:36:09.555416 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.555384    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"]
Jul 30 20:36:09.563363 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.563341    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:09.565988 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.565966    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fl9qc\""
Jul 30 20:36:09.566167 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.566109    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 20:36:09.566910 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.566887    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"]
Jul 30 20:36:09.641336 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.641306    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"]
Jul 30 20:36:09.657902 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.657878    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5bb8b39d-e702-4fcc-b240-c82b1d251043-config-file\") pod \"limitador-limitador-859c4d96d6-5bqn6\" (UID: \"5bb8b39d-e702-4fcc-b240-c82b1d251043\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:09.658008 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.657907    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jszgz\" (UniqueName: \"kubernetes.io/projected/5bb8b39d-e702-4fcc-b240-c82b1d251043-kube-api-access-jszgz\") pod \"limitador-limitador-859c4d96d6-5bqn6\" (UID: \"5bb8b39d-e702-4fcc-b240-c82b1d251043\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:09.759039 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.759009    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5bb8b39d-e702-4fcc-b240-c82b1d251043-config-file\") pod \"limitador-limitador-859c4d96d6-5bqn6\" (UID: \"5bb8b39d-e702-4fcc-b240-c82b1d251043\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:09.759163 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.759042    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jszgz\" (UniqueName: \"kubernetes.io/projected/5bb8b39d-e702-4fcc-b240-c82b1d251043-kube-api-access-jszgz\") pod \"limitador-limitador-859c4d96d6-5bqn6\" (UID: \"5bb8b39d-e702-4fcc-b240-c82b1d251043\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:09.759607 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.759588    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5bb8b39d-e702-4fcc-b240-c82b1d251043-config-file\") pod \"limitador-limitador-859c4d96d6-5bqn6\" (UID: \"5bb8b39d-e702-4fcc-b240-c82b1d251043\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:09.767163 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.767142    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jszgz\" (UniqueName: \"kubernetes.io/projected/5bb8b39d-e702-4fcc-b240-c82b1d251043-kube-api-access-jszgz\") pod \"limitador-limitador-859c4d96d6-5bqn6\" (UID: \"5bb8b39d-e702-4fcc-b240-c82b1d251043\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:09.875271 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:09.875245    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:10.016429 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:10.016402    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"]
Jul 30 20:36:10.019780 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:36:10.019755    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb8b39d_e702_4fcc_b240_c82b1d251043.slice/crio-22d2539d3c6d23c9ce668ecb01ffba2c46e55bbb609dd431af53411b83b8ff08 WatchSource:0}: Error finding container 22d2539d3c6d23c9ce668ecb01ffba2c46e55bbb609dd431af53411b83b8ff08: Status 404 returned error can't find the container with id 22d2539d3c6d23c9ce668ecb01ffba2c46e55bbb609dd431af53411b83b8ff08
Jul 30 20:36:10.723642 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:10.723610    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6" event={"ID":"5bb8b39d-e702-4fcc-b240-c82b1d251043","Type":"ContainerStarted","Data":"22d2539d3c6d23c9ce668ecb01ffba2c46e55bbb609dd431af53411b83b8ff08"}
Jul 30 20:36:12.735350 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:12.735301    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6" event={"ID":"5bb8b39d-e702-4fcc-b240-c82b1d251043","Type":"ContainerStarted","Data":"8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d"}
Jul 30 20:36:12.735832 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:12.735431    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:12.756737 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:12.755686    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6" podStartSLOduration=1.907558667 podStartE2EDuration="3.755665542s" totalImagesPullingTime="1.848106875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:36:12.752226284 +0000 UTC m=+651.213185098" watchObservedRunningTime="2026-07-30 20:36:12.755665542 +0000 UTC m=+651.216624356"
Jul 30 20:36:23.740983 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:23.740950    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:24.583799 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:24.583762    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"]
Jul 30 20:36:24.584388 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:24.584330    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6" podUID="5bb8b39d-e702-4fcc-b240-c82b1d251043" containerName="limitador" containerID="cri-o://8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d" gracePeriod=30
Jul 30 20:36:25.139977 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.139952    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:25.178784 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.178729    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jszgz\" (UniqueName: \"kubernetes.io/projected/5bb8b39d-e702-4fcc-b240-c82b1d251043-kube-api-access-jszgz\") pod \"5bb8b39d-e702-4fcc-b240-c82b1d251043\" (UID: \"5bb8b39d-e702-4fcc-b240-c82b1d251043\") "
Jul 30 20:36:25.178886 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.178802    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5bb8b39d-e702-4fcc-b240-c82b1d251043-config-file\") pod \"5bb8b39d-e702-4fcc-b240-c82b1d251043\" (UID: \"5bb8b39d-e702-4fcc-b240-c82b1d251043\") "
Jul 30 20:36:25.179184 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.179160    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bb8b39d-e702-4fcc-b240-c82b1d251043-config-file" (OuterVolumeSpecName: "config-file") pod "5bb8b39d-e702-4fcc-b240-c82b1d251043" (UID: "5bb8b39d-e702-4fcc-b240-c82b1d251043"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:36:25.180934 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.180912    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bb8b39d-e702-4fcc-b240-c82b1d251043-kube-api-access-jszgz" (OuterVolumeSpecName: "kube-api-access-jszgz") pod "5bb8b39d-e702-4fcc-b240-c82b1d251043" (UID: "5bb8b39d-e702-4fcc-b240-c82b1d251043"). InnerVolumeSpecName "kube-api-access-jszgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:36:25.280251 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.280224    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jszgz\" (UniqueName: \"kubernetes.io/projected/5bb8b39d-e702-4fcc-b240-c82b1d251043-kube-api-access-jszgz\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:36:25.280251 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.280247    2547 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5bb8b39d-e702-4fcc-b240-c82b1d251043-config-file\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:36:25.801006 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.800972    2547 generic.go:356] "Generic (PLEG): container finished" podID="5bb8b39d-e702-4fcc-b240-c82b1d251043" containerID="8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d" exitCode=0
Jul 30 20:36:25.801171 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.801024    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"
Jul 30 20:36:25.801171 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.801054    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6" event={"ID":"5bb8b39d-e702-4fcc-b240-c82b1d251043","Type":"ContainerDied","Data":"8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d"}
Jul 30 20:36:25.801171 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.801110    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-5bqn6" event={"ID":"5bb8b39d-e702-4fcc-b240-c82b1d251043","Type":"ContainerDied","Data":"22d2539d3c6d23c9ce668ecb01ffba2c46e55bbb609dd431af53411b83b8ff08"}
Jul 30 20:36:25.801171 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.801132    2547 scope.go:117] "RemoveContainer" containerID="8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d"
Jul 30 20:36:25.811620 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.811601    2547 scope.go:117] "RemoveContainer" containerID="8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d"
Jul 30 20:36:25.811936 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:36:25.811912    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d\": container with ID starting with 8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d not found: ID does not exist" containerID="8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d"
Jul 30 20:36:25.812017 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.811948    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d"} err="failed to get container status \"8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d\": rpc error: code = NotFound desc = could not find container \"8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d\": container with ID starting with 8aec072c9bc024f689c3acf54cecc8d2eb24100270ed8e565c628bc4d845138d not found: ID does not exist"
Jul 30 20:36:25.826549 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.826521    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"]
Jul 30 20:36:25.832860 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:25.832839    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-5bqn6"]
Jul 30 20:36:26.017682 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.017648    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"]
Jul 30 20:36:26.018069 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.018051    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bb8b39d-e702-4fcc-b240-c82b1d251043" containerName="limitador"
Jul 30 20:36:26.018149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.018071    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bb8b39d-e702-4fcc-b240-c82b1d251043" containerName="limitador"
Jul 30 20:36:26.018199 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.018166    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bb8b39d-e702-4fcc-b240-c82b1d251043" containerName="limitador"
Jul 30 20:36:26.022392 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.022372    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:26.025824 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.025808    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 30 20:36:26.025967 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.025951    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 20:36:26.026078 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.026062    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 20:36:26.026459 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.026444    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-r7nv2\""
Jul 30 20:36:26.029718 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.029433    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"]
Jul 30 20:36:26.072113 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.072082    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bb8b39d-e702-4fcc-b240-c82b1d251043" path="/var/lib/kubelet/pods/5bb8b39d-e702-4fcc-b240-c82b1d251043/volumes"
Jul 30 20:36:26.086902 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.086875    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k28m7\" (UniqueName: \"kubernetes.io/projected/a7837f60-07d7-48e0-9a58-193f428bb5b3-kube-api-access-k28m7\") pod \"postgres-69fb4594cf-mhmtl\" (UID: \"a7837f60-07d7-48e0-9a58-193f428bb5b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:26.086995 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.086959    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a7837f60-07d7-48e0-9a58-193f428bb5b3-data\") pod \"postgres-69fb4594cf-mhmtl\" (UID: \"a7837f60-07d7-48e0-9a58-193f428bb5b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:26.187598 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.187575    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k28m7\" (UniqueName: \"kubernetes.io/projected/a7837f60-07d7-48e0-9a58-193f428bb5b3-kube-api-access-k28m7\") pod \"postgres-69fb4594cf-mhmtl\" (UID: \"a7837f60-07d7-48e0-9a58-193f428bb5b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:26.187925 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.187648    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a7837f60-07d7-48e0-9a58-193f428bb5b3-data\") pod \"postgres-69fb4594cf-mhmtl\" (UID: \"a7837f60-07d7-48e0-9a58-193f428bb5b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:26.188005 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.187988    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a7837f60-07d7-48e0-9a58-193f428bb5b3-data\") pod \"postgres-69fb4594cf-mhmtl\" (UID: \"a7837f60-07d7-48e0-9a58-193f428bb5b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:26.201640 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.201617    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k28m7\" (UniqueName: \"kubernetes.io/projected/a7837f60-07d7-48e0-9a58-193f428bb5b3-kube-api-access-k28m7\") pod \"postgres-69fb4594cf-mhmtl\" (UID: \"a7837f60-07d7-48e0-9a58-193f428bb5b3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:26.333914 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.333862    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:26.464899 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.464875    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"]
Jul 30 20:36:26.466651 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:36:26.466625    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7837f60_07d7_48e0_9a58_193f428bb5b3.slice/crio-549d1253fa9c5e44a8c99336335fe501cb3925273559f900c10e128addd689a9 WatchSource:0}: Error finding container 549d1253fa9c5e44a8c99336335fe501cb3925273559f900c10e128addd689a9: Status 404 returned error can't find the container with id 549d1253fa9c5e44a8c99336335fe501cb3925273559f900c10e128addd689a9
Jul 30 20:36:26.809498 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:26.809414    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl" event={"ID":"a7837f60-07d7-48e0-9a58-193f428bb5b3","Type":"ContainerStarted","Data":"549d1253fa9c5e44a8c99336335fe501cb3925273559f900c10e128addd689a9"}
Jul 30 20:36:31.844455 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:31.844425    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl" event={"ID":"a7837f60-07d7-48e0-9a58-193f428bb5b3","Type":"ContainerStarted","Data":"1aab515b100e842d0db205fbeae8c9adf56c3df3b6b58005f2761b4ee3beb64d"}
Jul 30 20:36:31.844828 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:31.844491    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:31.863049 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:31.863003    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl" podStartSLOduration=1.947352298 podStartE2EDuration="6.862989258s" totalImagesPullingTime="4.91563696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:36:31.861788771 +0000 UTC m=+670.322747586" watchObservedRunningTime="2026-07-30 20:36:31.862989258 +0000 UTC m=+670.323948070"
Jul 30 20:36:37.878543 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:37.878513    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mhmtl"
Jul 30 20:36:38.721747 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.721715    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69bbb8cd66-k5pnn"]
Jul 30 20:36:38.728059 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.728034    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:36:38.731891 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.731871    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 30 20:36:38.731997 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.731870    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t4n2x\""
Jul 30 20:36:38.737560 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.737533    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69bbb8cd66-k5pnn"]
Jul 30 20:36:38.793877 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.793851    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz6f8\" (UniqueName: \"kubernetes.io/projected/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-kube-api-access-nz6f8\") pod \"authorino-69bbb8cd66-k5pnn\" (UID: \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\") " pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:36:38.793975 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.793906    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-tls-cert\") pod \"authorino-69bbb8cd66-k5pnn\" (UID: \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\") " pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:36:38.894865 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.894838    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz6f8\" (UniqueName: \"kubernetes.io/projected/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-kube-api-access-nz6f8\") pod \"authorino-69bbb8cd66-k5pnn\" (UID: \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\") " pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:36:38.895219 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.894882    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-tls-cert\") pod \"authorino-69bbb8cd66-k5pnn\" (UID: \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\") " pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:36:38.897392 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.897365    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-tls-cert\") pod \"authorino-69bbb8cd66-k5pnn\" (UID: \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\") " pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:36:38.904972 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:38.904951    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz6f8\" (UniqueName: \"kubernetes.io/projected/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-kube-api-access-nz6f8\") pod \"authorino-69bbb8cd66-k5pnn\" (UID: \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\") " pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:36:39.037746 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:39.037668    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:36:39.170749 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:39.170723    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69bbb8cd66-k5pnn"]
Jul 30 20:36:39.173119 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:36:39.173088    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76aee1b3_6d11_4a5f_952e_eb3cd96b796f.slice/crio-5f74f51cc8066d12486fcbb3bf64599722363b009f68e136cbdc14291920d19e WatchSource:0}: Error finding container 5f74f51cc8066d12486fcbb3bf64599722363b009f68e136cbdc14291920d19e: Status 404 returned error can't find the container with id 5f74f51cc8066d12486fcbb3bf64599722363b009f68e136cbdc14291920d19e
Jul 30 20:36:39.889681 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:39.889586    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn" event={"ID":"76aee1b3-6d11-4a5f-952e-eb3cd96b796f","Type":"ContainerStarted","Data":"5f74f51cc8066d12486fcbb3bf64599722363b009f68e136cbdc14291920d19e"}
Jul 30 20:36:40.896932 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:40.896897    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn" event={"ID":"76aee1b3-6d11-4a5f-952e-eb3cd96b796f","Type":"ContainerStarted","Data":"f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa"}
Jul 30 20:36:40.916118 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:40.916060    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn" podStartSLOduration=1.310348762 podStartE2EDuration="2.91603867s" totalImagesPullingTime="1.605689908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:36:40.912681564 +0000 UTC m=+679.373640377" watchObservedRunningTime="2026-07-30 20:36:40.91603867 +0000 UTC m=+679.376997488"
Jul 30 20:36:45.567760 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.567728    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55bc447b6-7nwqj"]
Jul 30 20:36:45.571054 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.571037    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:36:45.574962 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.574939    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 20:36:45.574962 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.574953    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6mz44\""
Jul 30 20:36:45.575163 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.574959    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 20:36:45.580524 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.580502    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55bc447b6-7nwqj"]
Jul 30 20:36:45.648227 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.648201    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kckcl\" (UniqueName: \"kubernetes.io/projected/25523731-cddb-4ecc-bb0c-88cc1888d270-kube-api-access-kckcl\") pod \"maas-controller-55bc447b6-7nwqj\" (UID: \"25523731-cddb-4ecc-bb0c-88cc1888d270\") " pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:36:45.648350 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.648251    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25523731-cddb-4ecc-bb0c-88cc1888d270-webhook-cert\") pod \"maas-controller-55bc447b6-7nwqj\" (UID: \"25523731-cddb-4ecc-bb0c-88cc1888d270\") " pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:36:45.748590 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.748563    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kckcl\" (UniqueName: \"kubernetes.io/projected/25523731-cddb-4ecc-bb0c-88cc1888d270-kube-api-access-kckcl\") pod \"maas-controller-55bc447b6-7nwqj\" (UID: \"25523731-cddb-4ecc-bb0c-88cc1888d270\") " pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:36:45.748678 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.748615    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25523731-cddb-4ecc-bb0c-88cc1888d270-webhook-cert\") pod \"maas-controller-55bc447b6-7nwqj\" (UID: \"25523731-cddb-4ecc-bb0c-88cc1888d270\") " pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:36:45.751155 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.751130    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25523731-cddb-4ecc-bb0c-88cc1888d270-webhook-cert\") pod \"maas-controller-55bc447b6-7nwqj\" (UID: \"25523731-cddb-4ecc-bb0c-88cc1888d270\") " pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:36:45.756827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.756806    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kckcl\" (UniqueName: \"kubernetes.io/projected/25523731-cddb-4ecc-bb0c-88cc1888d270-kube-api-access-kckcl\") pod \"maas-controller-55bc447b6-7nwqj\" (UID: \"25523731-cddb-4ecc-bb0c-88cc1888d270\") " pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:36:45.882715 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:45.882683    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:36:46.013221 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:46.013197    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55bc447b6-7nwqj"]
Jul 30 20:36:46.015412 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:36:46.015376    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25523731_cddb_4ecc_bb0c_88cc1888d270.slice/crio-878177baa5927d493d668448c94b4b0ff6730d563b8edef7cd753a1d7430d3d8 WatchSource:0}: Error finding container 878177baa5927d493d668448c94b4b0ff6730d563b8edef7cd753a1d7430d3d8: Status 404 returned error can't find the container with id 878177baa5927d493d668448c94b4b0ff6730d563b8edef7cd753a1d7430d3d8
Jul 30 20:36:46.929643 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:46.929607    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55bc447b6-7nwqj" event={"ID":"25523731-cddb-4ecc-bb0c-88cc1888d270","Type":"ContainerStarted","Data":"878177baa5927d493d668448c94b4b0ff6730d563b8edef7cd753a1d7430d3d8"}
Jul 30 20:36:48.943143 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:48.943103    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55bc447b6-7nwqj" event={"ID":"25523731-cddb-4ecc-bb0c-88cc1888d270","Type":"ContainerStarted","Data":"304a237e0ad01792c16303e9a99cfe2b4511cc137e0844407f0e724f123d6456"}
Jul 30 20:36:48.943544 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:48.943187    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:36:48.960558 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:48.960505    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55bc447b6-7nwqj" podStartSLOduration=1.78289827 podStartE2EDuration="3.960487893s" totalImagesPullingTime="2.177589623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:36:48.960388102 +0000 UTC m=+687.421346913" watchObservedRunningTime="2026-07-30 20:36:48.960487893 +0000 UTC m=+687.421446705"
Jul 30 20:36:59.951577 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:36:59.951545    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:37:25.725404 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.725371    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"]
Jul 30 20:37:25.733374 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.733350    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:25.737625 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.737601    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jctkc\""
Jul 30 20:37:25.737797 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.737630    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 30 20:37:25.740126 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.740105    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"]
Jul 30 20:37:25.753129 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.753078    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89fed71f-29a0-4055-a6f2-0964f289bbe1-openshift-service-ca\") pod \"maas-api-697c5f7f45-bdgw5\" (UID: \"89fed71f-29a0-4055-a6f2-0964f289bbe1\") " pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:25.753252 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.753182    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46z7r\" (UniqueName: \"kubernetes.io/projected/89fed71f-29a0-4055-a6f2-0964f289bbe1-kube-api-access-46z7r\") pod \"maas-api-697c5f7f45-bdgw5\" (UID: \"89fed71f-29a0-4055-a6f2-0964f289bbe1\") " pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:25.753252 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.753222    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89fed71f-29a0-4055-a6f2-0964f289bbe1-maas-api-tls\") pod \"maas-api-697c5f7f45-bdgw5\" (UID: \"89fed71f-29a0-4055-a6f2-0964f289bbe1\") " pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:25.854614 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.854585    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89fed71f-29a0-4055-a6f2-0964f289bbe1-openshift-service-ca\") pod \"maas-api-697c5f7f45-bdgw5\" (UID: \"89fed71f-29a0-4055-a6f2-0964f289bbe1\") " pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:25.854775 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.854629    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46z7r\" (UniqueName: \"kubernetes.io/projected/89fed71f-29a0-4055-a6f2-0964f289bbe1-kube-api-access-46z7r\") pod \"maas-api-697c5f7f45-bdgw5\" (UID: \"89fed71f-29a0-4055-a6f2-0964f289bbe1\") " pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:25.854775 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.854647    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89fed71f-29a0-4055-a6f2-0964f289bbe1-maas-api-tls\") pod \"maas-api-697c5f7f45-bdgw5\" (UID: \"89fed71f-29a0-4055-a6f2-0964f289bbe1\") " pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:25.855281 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.855257    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89fed71f-29a0-4055-a6f2-0964f289bbe1-openshift-service-ca\") pod \"maas-api-697c5f7f45-bdgw5\" (UID: \"89fed71f-29a0-4055-a6f2-0964f289bbe1\") " pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:25.857210 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.857188    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89fed71f-29a0-4055-a6f2-0964f289bbe1-maas-api-tls\") pod \"maas-api-697c5f7f45-bdgw5\" (UID: \"89fed71f-29a0-4055-a6f2-0964f289bbe1\") " pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:25.863117 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:25.863093    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46z7r\" (UniqueName: \"kubernetes.io/projected/89fed71f-29a0-4055-a6f2-0964f289bbe1-kube-api-access-46z7r\") pod \"maas-api-697c5f7f45-bdgw5\" (UID: \"89fed71f-29a0-4055-a6f2-0964f289bbe1\") " pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:26.045430 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.045364    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:26.196841 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.196340    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"]
Jul 30 20:37:26.198867 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:37:26.198823    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89fed71f_29a0_4055_a6f2_0964f289bbe1.slice/crio-cc5648f3f1017467cee394c49ea6d42f8d83856b7a622fc81af1f31cb56c8643 WatchSource:0}: Error finding container cc5648f3f1017467cee394c49ea6d42f8d83856b7a622fc81af1f31cb56c8643: Status 404 returned error can't find the container with id cc5648f3f1017467cee394c49ea6d42f8d83856b7a622fc81af1f31cb56c8643
Jul 30 20:37:26.208272 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.205999    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"]
Jul 30 20:37:26.211241 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.211218    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:37:26.214055 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.214032    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 30 20:37:26.214171 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.214076    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4bzq9\""
Jul 30 20:37:26.219176 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.218943    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"]
Jul 30 20:37:26.257802 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.257772    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74572307-3be4-4d10-a667-a7ccfbf839ba-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-56rxl\" (UID: \"74572307-3be4-4d10-a667-a7ccfbf839ba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:37:26.258004 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.257866    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff2k7\" (UniqueName: \"kubernetes.io/projected/74572307-3be4-4d10-a667-a7ccfbf839ba-kube-api-access-ff2k7\") pod \"payload-pre-processing-687dbb6cf6-56rxl\" (UID: \"74572307-3be4-4d10-a667-a7ccfbf839ba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:37:26.359412 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.359366    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff2k7\" (UniqueName: \"kubernetes.io/projected/74572307-3be4-4d10-a667-a7ccfbf839ba-kube-api-access-ff2k7\") pod \"payload-pre-processing-687dbb6cf6-56rxl\" (UID: \"74572307-3be4-4d10-a667-a7ccfbf839ba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:37:26.359588 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.359480    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74572307-3be4-4d10-a667-a7ccfbf839ba-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-56rxl\" (UID: \"74572307-3be4-4d10-a667-a7ccfbf839ba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:37:26.365633 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.365599    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74572307-3be4-4d10-a667-a7ccfbf839ba-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-56rxl\" (UID: \"74572307-3be4-4d10-a667-a7ccfbf839ba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:37:26.368433 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.368401    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff2k7\" (UniqueName: \"kubernetes.io/projected/74572307-3be4-4d10-a667-a7ccfbf839ba-kube-api-access-ff2k7\") pod \"payload-pre-processing-687dbb6cf6-56rxl\" (UID: \"74572307-3be4-4d10-a667-a7ccfbf839ba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:37:26.525602 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.525570    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:37:26.658350 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:26.658316    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"]
Jul 30 20:37:26.661420 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:37:26.661388    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74572307_3be4_4d10_a667_a7ccfbf839ba.slice/crio-8144cce40b0a7564e89235ae763696cbfa8a3d3fd9a081824d85796ad1010f92 WatchSource:0}: Error finding container 8144cce40b0a7564e89235ae763696cbfa8a3d3fd9a081824d85796ad1010f92: Status 404 returned error can't find the container with id 8144cce40b0a7564e89235ae763696cbfa8a3d3fd9a081824d85796ad1010f92
Jul 30 20:37:27.135277 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:27.135243    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5" event={"ID":"89fed71f-29a0-4055-a6f2-0964f289bbe1","Type":"ContainerStarted","Data":"cc5648f3f1017467cee394c49ea6d42f8d83856b7a622fc81af1f31cb56c8643"}
Jul 30 20:37:27.136724 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:27.136671    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl" event={"ID":"74572307-3be4-4d10-a667-a7ccfbf839ba","Type":"ContainerStarted","Data":"8144cce40b0a7564e89235ae763696cbfa8a3d3fd9a081824d85796ad1010f92"}
Jul 30 20:37:29.153228 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:29.153192    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5" event={"ID":"89fed71f-29a0-4055-a6f2-0964f289bbe1","Type":"ContainerStarted","Data":"bd17c5ce15c09ad7518af38a2ac94d5290381ceec5e030a28274730e558d86cb"}
Jul 30 20:37:29.153827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:29.153298    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:29.154757 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:29.154733    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl" event={"ID":"74572307-3be4-4d10-a667-a7ccfbf839ba","Type":"ContainerStarted","Data":"ae2d3ce1c7a777066e1989390e561718be61ea9002b2b2757c846224a5ae8369"}
Jul 30 20:37:29.154900 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:29.154778    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:37:29.169348 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:29.169307    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5" podStartSLOduration=2.282041344 podStartE2EDuration="4.169294614s" totalImagesPullingTime="1.88725327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:37:29.168653013 +0000 UTC m=+727.629611827" watchObservedRunningTime="2026-07-30 20:37:29.169294614 +0000 UTC m=+727.630253462"
Jul 30 20:37:29.183218 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:29.183183    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl" podStartSLOduration=1.367362275 podStartE2EDuration="3.183169543s" totalImagesPullingTime="1.815807268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:37:29.182165415 +0000 UTC m=+727.643124228" watchObservedRunningTime="2026-07-30 20:37:29.183169543 +0000 UTC m=+727.644128355"
Jul 30 20:37:35.165021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:35.164990    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-697c5f7f45-bdgw5"
Jul 30 20:37:40.162139 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:37:40.162104    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-56rxl"
Jul 30 20:38:15.143638 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.143606    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"]
Jul 30 20:38:15.148820 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.148801    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.151898 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.151873    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 20:38:15.152007 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.151872    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 30 20:38:15.152007 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.151912    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 20:38:15.153116 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.153097    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6wjc2\""
Jul 30 20:38:15.156884 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.156863    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"]
Jul 30 20:38:15.269388 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.269365    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.269542 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.269399    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.269542 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.269418    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fht6q\" (UniqueName: \"kubernetes.io/projected/3f113f75-182b-4e0d-a80b-aa068d2e222e-kube-api-access-fht6q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.269662 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.269560    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.269662 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.269609    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f113f75-182b-4e0d-a80b-aa068d2e222e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.269785 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.269675    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.370298 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.370264    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.370432 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.370309    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f113f75-182b-4e0d-a80b-aa068d2e222e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.370432 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.370347    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.370432 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.370396    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.370597 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.370434    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.370597 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.370461    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fht6q\" (UniqueName: \"kubernetes.io/projected/3f113f75-182b-4e0d-a80b-aa068d2e222e-kube-api-access-fht6q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.370721 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.370668    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.370817 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.370787    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.370943 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.370847    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.372695 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.372671    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f113f75-182b-4e0d-a80b-aa068d2e222e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.373154 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.373133    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f113f75-182b-4e0d-a80b-aa068d2e222e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.378363 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.378344    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fht6q\" (UniqueName: \"kubernetes.io/projected/3f113f75-182b-4e0d-a80b-aa068d2e222e-kube-api-access-fht6q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6l46d\" (UID: \"3f113f75-182b-4e0d-a80b-aa068d2e222e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.460762 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.460684    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:15.593898 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.593874    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"]
Jul 30 20:38:15.596473 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:38:15.596442    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f113f75_182b_4e0d_a80b_aa068d2e222e.slice/crio-753d2c7e3654143b78aa22aed33f9a7f0e287ac25109852d435f5e7f0531104f WatchSource:0}: Error finding container 753d2c7e3654143b78aa22aed33f9a7f0e287ac25109852d435f5e7f0531104f: Status 404 returned error can't find the container with id 753d2c7e3654143b78aa22aed33f9a7f0e287ac25109852d435f5e7f0531104f
Jul 30 20:38:15.598020 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:15.597993    2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:38:16.381822 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:16.381785    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d" event={"ID":"3f113f75-182b-4e0d-a80b-aa068d2e222e","Type":"ContainerStarted","Data":"753d2c7e3654143b78aa22aed33f9a7f0e287ac25109852d435f5e7f0531104f"}
Jul 30 20:38:17.388655 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:17.388621    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d" event={"ID":"3f113f75-182b-4e0d-a80b-aa068d2e222e","Type":"ContainerStarted","Data":"62e2249dae26eff28b65cf825d64731f715b3956b9885ac644b3142a2fd44463"}
Jul 30 20:38:17.407064 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:17.407012    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d" podStartSLOduration=1.417188334 podStartE2EDuration="2.406995254s" totalImagesPullingTime="989.80692ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:38:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:38:17.406078582 +0000 UTC m=+775.867037408" watchObservedRunningTime="2026-07-30 20:38:17.406995254 +0000 UTC m=+775.867954066"
Jul 30 20:38:25.461089 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:25.461049    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:25.461569 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:25.461128    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:25.473671 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:25.473648    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:26.447124 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:26.447097    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6l46d"
Jul 30 20:38:30.639112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.639083    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"]
Jul 30 20:38:30.644018 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.643994    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.646678 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.646656    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 30 20:38:30.651559 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.651426    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"]
Jul 30 20:38:30.698349 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.698322    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.698486 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.698382    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.698486 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.698409    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r59tb\" (UniqueName: \"kubernetes.io/projected/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-kube-api-access-r59tb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.698486 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.698434    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.698618 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.698512    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.698618 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.698544    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.799010 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.798988    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.799144 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.799052    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.799144 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.799076    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r59tb\" (UniqueName: \"kubernetes.io/projected/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-kube-api-access-r59tb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.799144 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.799098    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.799144 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.799131    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.799369 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.799151    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.799426 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.799408    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.799478 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.799451    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.799519 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.799506    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.801556 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.801536    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.801919 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.801901    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.806943 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.806922    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r59tb\" (UniqueName: \"kubernetes.io/projected/bd0c2c7a-902c-4b86-a157-b4ca0ab7a665-kube-api-access-r59tb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k9dmh\" (UID: \"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:30.955133 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:30.955080    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:31.090297 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:31.090250    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"]
Jul 30 20:38:31.092484 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:38:31.092460    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0c2c7a_902c_4b86_a157_b4ca0ab7a665.slice/crio-d541d918eff7238971d8a05f80dcc9f4229f577ef78672df3ef8c92b27bd2885 WatchSource:0}: Error finding container d541d918eff7238971d8a05f80dcc9f4229f577ef78672df3ef8c92b27bd2885: Status 404 returned error can't find the container with id d541d918eff7238971d8a05f80dcc9f4229f577ef78672df3ef8c92b27bd2885
Jul 30 20:38:31.459820 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:31.459772    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh" event={"ID":"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665","Type":"ContainerStarted","Data":"a1062df8f8e2a467964cf6a0444929082b4ff0c6334a0611949eb34b698398b1"}
Jul 30 20:38:31.459985 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:31.459829    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh" event={"ID":"bd0c2c7a-902c-4b86-a157-b4ca0ab7a665","Type":"ContainerStarted","Data":"d541d918eff7238971d8a05f80dcc9f4229f577ef78672df3ef8c92b27bd2885"}
Jul 30 20:38:31.480133 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:31.480084    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh" podStartSLOduration=1.29821884 podStartE2EDuration="1.480069316s" totalImagesPullingTime="181.850476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:38:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:38:31.479250921 +0000 UTC m=+789.940209733" watchObservedRunningTime="2026-07-30 20:38:31.480069316 +0000 UTC m=+789.941028127"
Jul 30 20:38:40.955752 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:40.955695    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:40.956218 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:40.955835    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:40.967960 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:40.967934    2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:38:41.523663 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:38:41.523635    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k9dmh"
Jul 30 20:39:55.793981 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:55.793946    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-db9bdb8cf-fhblv"]
Jul 30 20:39:55.798192 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:55.798162    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-db9bdb8cf-fhblv"
Jul 30 20:39:55.806185 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:55.806155    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-db9bdb8cf-fhblv"]
Jul 30 20:39:55.923012 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:55.922980    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbfn4\" (UniqueName: \"kubernetes.io/projected/e5cf6ad1-708e-4c06-a744-5d6c69f1893d-kube-api-access-bbfn4\") pod \"authorino-db9bdb8cf-fhblv\" (UID: \"e5cf6ad1-708e-4c06-a744-5d6c69f1893d\") " pod="kuadrant-system/authorino-db9bdb8cf-fhblv"
Jul 30 20:39:55.923178 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:55.923056    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5cf6ad1-708e-4c06-a744-5d6c69f1893d-tls-cert\") pod \"authorino-db9bdb8cf-fhblv\" (UID: \"e5cf6ad1-708e-4c06-a744-5d6c69f1893d\") " pod="kuadrant-system/authorino-db9bdb8cf-fhblv"
Jul 30 20:39:56.024275 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:56.024244    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbfn4\" (UniqueName: \"kubernetes.io/projected/e5cf6ad1-708e-4c06-a744-5d6c69f1893d-kube-api-access-bbfn4\") pod \"authorino-db9bdb8cf-fhblv\" (UID: \"e5cf6ad1-708e-4c06-a744-5d6c69f1893d\") " pod="kuadrant-system/authorino-db9bdb8cf-fhblv"
Jul 30 20:39:56.024383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:56.024307    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5cf6ad1-708e-4c06-a744-5d6c69f1893d-tls-cert\") pod \"authorino-db9bdb8cf-fhblv\" (UID: \"e5cf6ad1-708e-4c06-a744-5d6c69f1893d\") " pod="kuadrant-system/authorino-db9bdb8cf-fhblv"
Jul 30 20:39:56.026922 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:56.026901    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5cf6ad1-708e-4c06-a744-5d6c69f1893d-tls-cert\") pod \"authorino-db9bdb8cf-fhblv\" (UID: \"e5cf6ad1-708e-4c06-a744-5d6c69f1893d\") " pod="kuadrant-system/authorino-db9bdb8cf-fhblv"
Jul 30 20:39:56.032847 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:56.032820    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbfn4\" (UniqueName: \"kubernetes.io/projected/e5cf6ad1-708e-4c06-a744-5d6c69f1893d-kube-api-access-bbfn4\") pod \"authorino-db9bdb8cf-fhblv\" (UID: \"e5cf6ad1-708e-4c06-a744-5d6c69f1893d\") " pod="kuadrant-system/authorino-db9bdb8cf-fhblv"
Jul 30 20:39:56.111368 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:56.111341    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-db9bdb8cf-fhblv"
Jul 30 20:39:56.243511 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:56.243489    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-db9bdb8cf-fhblv"]
Jul 30 20:39:56.245251 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:39:56.245228    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5cf6ad1_708e_4c06_a744_5d6c69f1893d.slice/crio-cfcdc9c9b08bcb1ac4eda74fd3101d453f49b0c510ef1e42a81598124f40da4a WatchSource:0}: Error finding container cfcdc9c9b08bcb1ac4eda74fd3101d453f49b0c510ef1e42a81598124f40da4a: Status 404 returned error can't find the container with id cfcdc9c9b08bcb1ac4eda74fd3101d453f49b0c510ef1e42a81598124f40da4a
Jul 30 20:39:56.889504 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:56.889473    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-db9bdb8cf-fhblv" event={"ID":"e5cf6ad1-708e-4c06-a744-5d6c69f1893d","Type":"ContainerStarted","Data":"cfcdc9c9b08bcb1ac4eda74fd3101d453f49b0c510ef1e42a81598124f40da4a"}
Jul 30 20:39:58.903046 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:58.903004    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-db9bdb8cf-fhblv" event={"ID":"e5cf6ad1-708e-4c06-a744-5d6c69f1893d","Type":"ContainerStarted","Data":"ba1987d46f6d65992ef2c84749af1bb80cf6f2d89f34d08ebca0572446c92086"}
Jul 30 20:39:58.924901 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:58.924847    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-db9bdb8cf-fhblv" podStartSLOduration=1.764599913 podStartE2EDuration="3.924831356s" totalImagesPullingTime="2.160231443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:39:58.922367472 +0000 UTC m=+877.383326286" watchObservedRunningTime="2026-07-30 20:39:58.924831356 +0000 UTC m=+877.385790168"
Jul 30 20:39:58.952651 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:58.952616    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69bbb8cd66-k5pnn"]
Jul 30 20:39:58.952891 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:58.952861    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn" podUID="76aee1b3-6d11-4a5f-952e-eb3cd96b796f" containerName="authorino" containerID="cri-o://f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa" gracePeriod=30
Jul 30 20:39:59.239411 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.239385    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:39:59.252862 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.252830    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-tls-cert\") pod \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\" (UID: \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\") "
Jul 30 20:39:59.252991 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.252890    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz6f8\" (UniqueName: \"kubernetes.io/projected/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-kube-api-access-nz6f8\") pod \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\" (UID: \"76aee1b3-6d11-4a5f-952e-eb3cd96b796f\") "
Jul 30 20:39:59.255410 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.255355    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-kube-api-access-nz6f8" (OuterVolumeSpecName: "kube-api-access-nz6f8") pod "76aee1b3-6d11-4a5f-952e-eb3cd96b796f" (UID: "76aee1b3-6d11-4a5f-952e-eb3cd96b796f"). InnerVolumeSpecName "kube-api-access-nz6f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:39:59.267171 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.267137    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "76aee1b3-6d11-4a5f-952e-eb3cd96b796f" (UID: "76aee1b3-6d11-4a5f-952e-eb3cd96b796f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:39:59.354361 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.354316    2547 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-tls-cert\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:39:59.354361 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.354352    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz6f8\" (UniqueName: \"kubernetes.io/projected/76aee1b3-6d11-4a5f-952e-eb3cd96b796f-kube-api-access-nz6f8\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:39:59.910550 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.910521    2547 generic.go:356] "Generic (PLEG): container finished" podID="76aee1b3-6d11-4a5f-952e-eb3cd96b796f" containerID="f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa" exitCode=0
Jul 30 20:39:59.911038 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.910575    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn"
Jul 30 20:39:59.911038 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.910614    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn" event={"ID":"76aee1b3-6d11-4a5f-952e-eb3cd96b796f","Type":"ContainerDied","Data":"f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa"}
Jul 30 20:39:59.911038 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.910670    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69bbb8cd66-k5pnn" event={"ID":"76aee1b3-6d11-4a5f-952e-eb3cd96b796f","Type":"ContainerDied","Data":"5f74f51cc8066d12486fcbb3bf64599722363b009f68e136cbdc14291920d19e"}
Jul 30 20:39:59.911038 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.910711    2547 scope.go:117] "RemoveContainer" containerID="f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa"
Jul 30 20:39:59.920027 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.920005    2547 scope.go:117] "RemoveContainer" containerID="f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa"
Jul 30 20:39:59.920294 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:39:59.920276    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa\": container with ID starting with f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa not found: ID does not exist" containerID="f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa"
Jul 30 20:39:59.920370 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.920307    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa"} err="failed to get container status \"f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa\": rpc error: code = NotFound desc = could not find container \"f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa\": container with ID starting with f9a8c2fc618dc0c9a17cd7bfaf489467dbe7ce1a31b929930cd6137ffbd06afa not found: ID does not exist"
Jul 30 20:39:59.935050 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.935018    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69bbb8cd66-k5pnn"]
Jul 30 20:39:59.943722 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:39:59.943686    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69bbb8cd66-k5pnn"]
Jul 30 20:40:00.075546 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:00.075509    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76aee1b3-6d11-4a5f-952e-eb3cd96b796f" path="/var/lib/kubelet/pods/76aee1b3-6d11-4a5f-952e-eb3cd96b796f/volumes"
Jul 30 20:40:22.033067 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:22.033044    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:40:22.035734 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:22.035714    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:40:22.037331 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:22.037310    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:40:22.039805 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:22.039786    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:40:47.004588 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.004550    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55bc447b6-7nwqj"]
Jul 30 20:40:47.007021 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.004824    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55bc447b6-7nwqj" podUID="25523731-cddb-4ecc-bb0c-88cc1888d270" containerName="manager" containerID="cri-o://304a237e0ad01792c16303e9a99cfe2b4511cc137e0844407f0e724f123d6456" gracePeriod=30
Jul 30 20:40:47.155341 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.155306    2547 generic.go:356] "Generic (PLEG): container finished" podID="25523731-cddb-4ecc-bb0c-88cc1888d270" containerID="304a237e0ad01792c16303e9a99cfe2b4511cc137e0844407f0e724f123d6456" exitCode=0
Jul 30 20:40:47.155475 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.155371    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55bc447b6-7nwqj" event={"ID":"25523731-cddb-4ecc-bb0c-88cc1888d270","Type":"ContainerDied","Data":"304a237e0ad01792c16303e9a99cfe2b4511cc137e0844407f0e724f123d6456"}
Jul 30 20:40:47.263017 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.262965    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:40:47.264233 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.264213    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kckcl\" (UniqueName: \"kubernetes.io/projected/25523731-cddb-4ecc-bb0c-88cc1888d270-kube-api-access-kckcl\") pod \"25523731-cddb-4ecc-bb0c-88cc1888d270\" (UID: \"25523731-cddb-4ecc-bb0c-88cc1888d270\") "
Jul 30 20:40:47.264309 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.264262    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25523731-cddb-4ecc-bb0c-88cc1888d270-webhook-cert\") pod \"25523731-cddb-4ecc-bb0c-88cc1888d270\" (UID: \"25523731-cddb-4ecc-bb0c-88cc1888d270\") "
Jul 30 20:40:47.266328 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.266300    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25523731-cddb-4ecc-bb0c-88cc1888d270-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "25523731-cddb-4ecc-bb0c-88cc1888d270" (UID: "25523731-cddb-4ecc-bb0c-88cc1888d270"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:40:47.266483 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.266377    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25523731-cddb-4ecc-bb0c-88cc1888d270-kube-api-access-kckcl" (OuterVolumeSpecName: "kube-api-access-kckcl") pod "25523731-cddb-4ecc-bb0c-88cc1888d270" (UID: "25523731-cddb-4ecc-bb0c-88cc1888d270"). InnerVolumeSpecName "kube-api-access-kckcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:40:47.364894 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.364865    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kckcl\" (UniqueName: \"kubernetes.io/projected/25523731-cddb-4ecc-bb0c-88cc1888d270-kube-api-access-kckcl\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:40:47.364894 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:47.364890    2547 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25523731-cddb-4ecc-bb0c-88cc1888d270-webhook-cert\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:40:48.161888 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:48.161853    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55bc447b6-7nwqj"
Jul 30 20:40:48.162265 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:48.161863    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55bc447b6-7nwqj" event={"ID":"25523731-cddb-4ecc-bb0c-88cc1888d270","Type":"ContainerDied","Data":"878177baa5927d493d668448c94b4b0ff6730d563b8edef7cd753a1d7430d3d8"}
Jul 30 20:40:48.162265 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:48.161989    2547 scope.go:117] "RemoveContainer" containerID="304a237e0ad01792c16303e9a99cfe2b4511cc137e0844407f0e724f123d6456"
Jul 30 20:40:48.178806 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:48.178782    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55bc447b6-7nwqj"]
Jul 30 20:40:48.182239 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:48.182214    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55bc447b6-7nwqj"]
Jul 30 20:40:50.073272 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:40:50.073241    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25523731-cddb-4ecc-bb0c-88cc1888d270" path="/var/lib/kubelet/pods/25523731-cddb-4ecc-bb0c-88cc1888d270/volumes"
Jul 30 20:45:00.139646 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.139569    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg"]
Jul 30 20:45:00.142745 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.140594    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76aee1b3-6d11-4a5f-952e-eb3cd96b796f" containerName="authorino"
Jul 30 20:45:00.142745 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.140634    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="76aee1b3-6d11-4a5f-952e-eb3cd96b796f" containerName="authorino"
Jul 30 20:45:00.142745 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.140777    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25523731-cddb-4ecc-bb0c-88cc1888d270" containerName="manager"
Jul 30 20:45:00.142745 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.140787    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="25523731-cddb-4ecc-bb0c-88cc1888d270" containerName="manager"
Jul 30 20:45:00.142745 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.141004    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="76aee1b3-6d11-4a5f-952e-eb3cd96b796f" containerName="authorino"
Jul 30 20:45:00.142745 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.141026    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="25523731-cddb-4ecc-bb0c-88cc1888d270" containerName="manager"
Jul 30 20:45:00.146684 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.146657    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg"
Jul 30 20:45:00.149837 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.149820    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zm2hs\""
Jul 30 20:45:00.156240 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.156222    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg"
Jul 30 20:45:00.162144 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.162111    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg"]
Jul 30 20:45:00.291606 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.291579    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg"]
Jul 30 20:45:00.292779 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:45:00.292756    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e997265_01b6_4575_999c_f16ec66d7966.slice/crio-992412d8eb9497f276edca7a8373af5d8f7062dbabc10b3f1572e5ed69def174 WatchSource:0}: Error finding container 992412d8eb9497f276edca7a8373af5d8f7062dbabc10b3f1572e5ed69def174: Status 404 returned error can't find the container with id 992412d8eb9497f276edca7a8373af5d8f7062dbabc10b3f1572e5ed69def174
Jul 30 20:45:00.294758 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.294737    2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:45:00.384470 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:00.384439    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg" event={"ID":"4e997265-01b6-4575-999c-f16ec66d7966","Type":"ContainerStarted","Data":"992412d8eb9497f276edca7a8373af5d8f7062dbabc10b3f1572e5ed69def174"}
Jul 30 20:45:03.400969 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:03.400937    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg" event={"ID":"4e997265-01b6-4575-999c-f16ec66d7966","Type":"ContainerStarted","Data":"f76f75c1d464c25d2751efd2e5b19f19653990edc211ad5a03c7f42b69c56494"}
Jul 30 20:45:03.416891 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:03.416851    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg" podStartSLOduration=0.66211634 podStartE2EDuration="3.416837612s" totalImagesPullingTime="2.754721272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:45:03.416168666 +0000 UTC m=+1181.877127477" watchObservedRunningTime="2026-07-30 20:45:03.416837612 +0000 UTC m=+1181.877796423"
Jul 30 20:45:04.408179 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:04.408151    2547 generic.go:356] "Generic (PLEG): container finished" podID="4e997265-01b6-4575-999c-f16ec66d7966" containerID="f76f75c1d464c25d2751efd2e5b19f19653990edc211ad5a03c7f42b69c56494" exitCode=0
Jul 30 20:45:04.408516 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:04.408204    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg" event={"ID":"4e997265-01b6-4575-999c-f16ec66d7966","Type":"ContainerDied","Data":"f76f75c1d464c25d2751efd2e5b19f19653990edc211ad5a03c7f42b69c56494"}
Jul 30 20:45:05.547732 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:05.547694    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg"
Jul 30 20:45:06.419534 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:06.419463    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg" event={"ID":"4e997265-01b6-4575-999c-f16ec66d7966","Type":"ContainerDied","Data":"992412d8eb9497f276edca7a8373af5d8f7062dbabc10b3f1572e5ed69def174"}
Jul 30 20:45:06.419534 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:06.419508    2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="992412d8eb9497f276edca7a8373af5d8f7062dbabc10b3f1572e5ed69def174"
Jul 30 20:45:06.419534 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:06.419509    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757405-ghlmg"
Jul 30 20:45:22.063635 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:22.063607    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:45:22.066089 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:22.066061    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:45:22.068165 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:22.068143    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:45:22.070579 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:45:22.070555    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:48:37.571955 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:37.571921    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"]
Jul 30 20:48:37.574462 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:37.572099    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt" podUID="3536195d-3a77-4aed-b715-fb7d0ac7c84b" containerName="manager" containerID="cri-o://2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d" gracePeriod=10
Jul 30 20:48:37.821693 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:37.821667    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:48:37.999947 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:37.999871    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgg58\" (UniqueName: \"kubernetes.io/projected/3536195d-3a77-4aed-b715-fb7d0ac7c84b-kube-api-access-vgg58\") pod \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\" (UID: \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\") "
Jul 30 20:48:38.000100 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:37.999973    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3536195d-3a77-4aed-b715-fb7d0ac7c84b-extensions-socket-volume\") pod \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\" (UID: \"3536195d-3a77-4aed-b715-fb7d0ac7c84b\") "
Jul 30 20:48:38.000319 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.000291    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3536195d-3a77-4aed-b715-fb7d0ac7c84b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3536195d-3a77-4aed-b715-fb7d0ac7c84b" (UID: "3536195d-3a77-4aed-b715-fb7d0ac7c84b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:48:38.001873 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.001845    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3536195d-3a77-4aed-b715-fb7d0ac7c84b-kube-api-access-vgg58" (OuterVolumeSpecName: "kube-api-access-vgg58") pod "3536195d-3a77-4aed-b715-fb7d0ac7c84b" (UID: "3536195d-3a77-4aed-b715-fb7d0ac7c84b"). InnerVolumeSpecName "kube-api-access-vgg58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:48:38.100908 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.100883    2547 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3536195d-3a77-4aed-b715-fb7d0ac7c84b-extensions-socket-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:48:38.100908 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.100904    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgg58\" (UniqueName: \"kubernetes.io/projected/3536195d-3a77-4aed-b715-fb7d0ac7c84b-kube-api-access-vgg58\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:48:38.434082 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.434057    2547 generic.go:356] "Generic (PLEG): container finished" podID="3536195d-3a77-4aed-b715-fb7d0ac7c84b" containerID="2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d" exitCode=0
Jul 30 20:48:38.434189 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.434101    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"
Jul 30 20:48:38.434189 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.434128    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt" event={"ID":"3536195d-3a77-4aed-b715-fb7d0ac7c84b","Type":"ContainerDied","Data":"2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d"}
Jul 30 20:48:38.434189 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.434163    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt" event={"ID":"3536195d-3a77-4aed-b715-fb7d0ac7c84b","Type":"ContainerDied","Data":"b349fc072cabd492dd3d3b1da2be9336c15e87c8d5959dd19e3805c89e9d1575"}
Jul 30 20:48:38.434189 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.434183    2547 scope.go:117] "RemoveContainer" containerID="2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d"
Jul 30 20:48:38.444582 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.444564    2547 scope.go:117] "RemoveContainer" containerID="2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d"
Jul 30 20:48:38.444944 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:48:38.444925    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d\": container with ID starting with 2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d not found: ID does not exist" containerID="2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d"
Jul 30 20:48:38.444995 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.444957    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d"} err="failed to get container status \"2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d\": rpc error: code = NotFound desc = could not find container \"2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d\": container with ID starting with 2da035abd11d964d912e534b5f83ca1db03315eaa6cb9ec316565a6d810a338d not found: ID does not exist"
Jul 30 20:48:38.452908 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.452887    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"]
Jul 30 20:48:38.456657 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:38.456637    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-n4bqt"]
Jul 30 20:48:40.072180 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:48:40.072146    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3536195d-3a77-4aed-b715-fb7d0ac7c84b" path="/var/lib/kubelet/pods/3536195d-3a77-4aed-b715-fb7d0ac7c84b/volumes"
Jul 30 20:49:38.285827 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.285789    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"]
Jul 30 20:49:38.286367 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.286329    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e997265-01b6-4575-999c-f16ec66d7966" containerName="cleanup"
Jul 30 20:49:38.286367 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.286348    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e997265-01b6-4575-999c-f16ec66d7966" containerName="cleanup"
Jul 30 20:49:38.286367 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.286360    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3536195d-3a77-4aed-b715-fb7d0ac7c84b" containerName="manager"
Jul 30 20:49:38.286534 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.286370    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="3536195d-3a77-4aed-b715-fb7d0ac7c84b" containerName="manager"
Jul 30 20:49:38.286534 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.286454    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="3536195d-3a77-4aed-b715-fb7d0ac7c84b" containerName="manager"
Jul 30 20:49:38.286534 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.286467    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e997265-01b6-4575-999c-f16ec66d7966" containerName="cleanup"
Jul 30 20:49:38.292054 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.292027    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:49:38.295771 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.295750    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4j9mr\""
Jul 30 20:49:38.302761 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.302738    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"]
Jul 30 20:49:38.362561 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.362539    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9268292-7944-4e0a-81fa-1d955c58878c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-jhnfd\" (UID: \"c9268292-7944-4e0a-81fa-1d955c58878c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:49:38.362665 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.362603    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph97l\" (UniqueName: \"kubernetes.io/projected/c9268292-7944-4e0a-81fa-1d955c58878c-kube-api-access-ph97l\") pod \"kuadrant-operator-controller-manager-68cf589589-jhnfd\" (UID: \"c9268292-7944-4e0a-81fa-1d955c58878c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:49:38.462958 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.462932    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph97l\" (UniqueName: \"kubernetes.io/projected/c9268292-7944-4e0a-81fa-1d955c58878c-kube-api-access-ph97l\") pod \"kuadrant-operator-controller-manager-68cf589589-jhnfd\" (UID: \"c9268292-7944-4e0a-81fa-1d955c58878c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:49:38.463045 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.462979    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9268292-7944-4e0a-81fa-1d955c58878c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-jhnfd\" (UID: \"c9268292-7944-4e0a-81fa-1d955c58878c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:49:38.463334 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.463319    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9268292-7944-4e0a-81fa-1d955c58878c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-jhnfd\" (UID: \"c9268292-7944-4e0a-81fa-1d955c58878c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:49:38.471788 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.471756    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph97l\" (UniqueName: \"kubernetes.io/projected/c9268292-7944-4e0a-81fa-1d955c58878c-kube-api-access-ph97l\") pod \"kuadrant-operator-controller-manager-68cf589589-jhnfd\" (UID: \"c9268292-7944-4e0a-81fa-1d955c58878c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:49:38.602332 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.602312    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:49:38.750506 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:38.750482    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"]
Jul 30 20:49:38.752633 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:49:38.752609    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9268292_7944_4e0a_81fa_1d955c58878c.slice/crio-569987b2648e761ec9ba5d63ad58098b87abb8f330a65258fade3f13859ce710 WatchSource:0}: Error finding container 569987b2648e761ec9ba5d63ad58098b87abb8f330a65258fade3f13859ce710: Status 404 returned error can't find the container with id 569987b2648e761ec9ba5d63ad58098b87abb8f330a65258fade3f13859ce710
Jul 30 20:49:39.725059 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:39.725025    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd" event={"ID":"c9268292-7944-4e0a-81fa-1d955c58878c","Type":"ContainerStarted","Data":"9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b"}
Jul 30 20:49:39.725473 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:39.725087    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd" event={"ID":"c9268292-7944-4e0a-81fa-1d955c58878c","Type":"ContainerStarted","Data":"569987b2648e761ec9ba5d63ad58098b87abb8f330a65258fade3f13859ce710"}
Jul 30 20:49:39.725473 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:39.725119    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:49:39.748072 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:39.748027    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd" podStartSLOduration=1.748013073 podStartE2EDuration="1.748013073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:49:39.745240908 +0000 UTC m=+1458.206199755" watchObservedRunningTime="2026-07-30 20:49:39.748013073 +0000 UTC m=+1458.208971883"
Jul 30 20:49:50.731299 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:49:50.731272    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 20:50:22.094791 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:50:22.094762    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:50:22.096402 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:50:22.096382    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:50:22.098659 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:50:22.098642    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:50:22.100397 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:50:22.100382    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:55:22.125467 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:55:22.125437    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:55:22.133983 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:55:22.133946    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 20:55:22.134842 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:55:22.134813    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:55:22.138428 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:55:22.138406    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 20:57:44.450563 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.450513    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"]
Jul 30 20:57:44.454375 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.454357    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.457359 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.457337    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-83e97498-serving-cert\""
Jul 30 20:57:44.474073 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.474041    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"]
Jul 30 20:57:44.629788 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.629751    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb7cb\" (UniqueName: \"kubernetes.io/projected/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-kube-api-access-sb7cb\") pod \"maas-api-e2e-ait-83e97498-585c89d855-nfkxv\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.629925 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.629814    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-openshift-service-ca\") pod \"maas-api-e2e-ait-83e97498-585c89d855-nfkxv\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.629925 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.629879    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-maas-api-tls\") pod \"maas-api-e2e-ait-83e97498-585c89d855-nfkxv\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.731120 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.731054    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-openshift-service-ca\") pod \"maas-api-e2e-ait-83e97498-585c89d855-nfkxv\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.731120 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.731088    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-maas-api-tls\") pod \"maas-api-e2e-ait-83e97498-585c89d855-nfkxv\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.731297 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.731155    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb7cb\" (UniqueName: \"kubernetes.io/projected/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-kube-api-access-sb7cb\") pod \"maas-api-e2e-ait-83e97498-585c89d855-nfkxv\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.731724 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.731673    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-openshift-service-ca\") pod \"maas-api-e2e-ait-83e97498-585c89d855-nfkxv\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.733641 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.733624    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-maas-api-tls\") pod \"maas-api-e2e-ait-83e97498-585c89d855-nfkxv\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.742899 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.742874    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb7cb\" (UniqueName: \"kubernetes.io/projected/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-kube-api-access-sb7cb\") pod \"maas-api-e2e-ait-83e97498-585c89d855-nfkxv\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:44.771583 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:44.771560    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:45.118062 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.118029    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"]
Jul 30 20:57:45.121713 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.121678    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:45.124664 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.124633    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-83e97498\""
Jul 30 20:57:45.124814 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.124794    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-83e97498-dockercfg-84qjw\""
Jul 30 20:57:45.132603 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.132576    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"]
Jul 30 20:57:45.136122 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.135945    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"]
Jul 30 20:57:45.138622 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:57:45.138599    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eeddef6_2c0b_49d6_bf4c_b9849ad6e290.slice/crio-f733a1dd384b386d9290b9b75ed4f6790be8d1a20b42bf4bb539c91ef5e82d3b WatchSource:0}: Error finding container f733a1dd384b386d9290b9b75ed4f6790be8d1a20b42bf4bb539c91ef5e82d3b: Status 404 returned error can't find the container with id f733a1dd384b386d9290b9b75ed4f6790be8d1a20b42bf4bb539c91ef5e82d3b
Jul 30 20:57:45.139971 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.139951    2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:57:45.162082 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.162053    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"]
Jul 30 20:57:45.165929 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.165908    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:57:45.177391 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.177369    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"]
Jul 30 20:57:45.235080 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.235053    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac278b-a580-4701-a016-e783df9a9f50-plugins-config-volume\") pod \"payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg\" (UID: \"beac278b-a580-4701-a016-e783df9a9f50\") " pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:45.235210 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.235091    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxdqr\" (UniqueName: \"kubernetes.io/projected/beac278b-a580-4701-a016-e783df9a9f50-kube-api-access-lxdqr\") pod \"payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg\" (UID: \"beac278b-a580-4701-a016-e783df9a9f50\") " pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:45.336456 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.336430    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac278b-a580-4701-a016-e783df9a9f50-plugins-config-volume\") pod \"payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg\" (UID: \"beac278b-a580-4701-a016-e783df9a9f50\") " pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:45.336578 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.336463    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxdqr\" (UniqueName: \"kubernetes.io/projected/beac278b-a580-4701-a016-e783df9a9f50-kube-api-access-lxdqr\") pod \"payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg\" (UID: \"beac278b-a580-4701-a016-e783df9a9f50\") " pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:45.336578 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.336486    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrpb2\" (UniqueName: \"kubernetes.io/projected/247102b0-b35c-4e25-a388-86efa0ebefd6-kube-api-access-xrpb2\") pod \"payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx\" (UID: \"247102b0-b35c-4e25-a388-86efa0ebefd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:57:45.336578 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.336523    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/247102b0-b35c-4e25-a388-86efa0ebefd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx\" (UID: \"247102b0-b35c-4e25-a388-86efa0ebefd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:57:45.337137 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.337118    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac278b-a580-4701-a016-e783df9a9f50-plugins-config-volume\") pod \"payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg\" (UID: \"beac278b-a580-4701-a016-e783df9a9f50\") " pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:45.345203 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.345181    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxdqr\" (UniqueName: \"kubernetes.io/projected/beac278b-a580-4701-a016-e783df9a9f50-kube-api-access-lxdqr\") pod \"payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg\" (UID: \"beac278b-a580-4701-a016-e783df9a9f50\") " pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:45.435269 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.435212    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:45.437512 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.437491    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrpb2\" (UniqueName: \"kubernetes.io/projected/247102b0-b35c-4e25-a388-86efa0ebefd6-kube-api-access-xrpb2\") pod \"payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx\" (UID: \"247102b0-b35c-4e25-a388-86efa0ebefd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:57:45.437588 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.437540    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/247102b0-b35c-4e25-a388-86efa0ebefd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx\" (UID: \"247102b0-b35c-4e25-a388-86efa0ebefd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:57:45.438193 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.438177    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/247102b0-b35c-4e25-a388-86efa0ebefd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx\" (UID: \"247102b0-b35c-4e25-a388-86efa0ebefd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:57:45.447507 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.447488    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrpb2\" (UniqueName: \"kubernetes.io/projected/247102b0-b35c-4e25-a388-86efa0ebefd6-kube-api-access-xrpb2\") pod \"payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx\" (UID: \"247102b0-b35c-4e25-a388-86efa0ebefd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:57:45.477372 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.477347    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:57:45.577313 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.577277    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"]
Jul 30 20:57:45.579973 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:57:45.579949    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeac278b_a580_4701_a016_e783df9a9f50.slice/crio-d4c54c7faf30461d70b8320e58a747164ab88548b2f34db2e0dffb279ee04245 WatchSource:0}: Error finding container d4c54c7faf30461d70b8320e58a747164ab88548b2f34db2e0dffb279ee04245: Status 404 returned error can't find the container with id d4c54c7faf30461d70b8320e58a747164ab88548b2f34db2e0dffb279ee04245
Jul 30 20:57:45.625778 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:45.625754    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"]
Jul 30 20:57:45.628224 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:57:45.628197    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247102b0_b35c_4e25_a388_86efa0ebefd6.slice/crio-f5cdefd0e5c896a19881ae56bb1be34d1bf02c5c7feb94c832200bf3114280a8 WatchSource:0}: Error finding container f5cdefd0e5c896a19881ae56bb1be34d1bf02c5c7feb94c832200bf3114280a8: Status 404 returned error can't find the container with id f5cdefd0e5c896a19881ae56bb1be34d1bf02c5c7feb94c832200bf3114280a8
Jul 30 20:57:46.046416 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.046321    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg" event={"ID":"beac278b-a580-4701-a016-e783df9a9f50","Type":"ContainerStarted","Data":"bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360"}
Jul 30 20:57:46.046416 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.046385    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg" event={"ID":"beac278b-a580-4701-a016-e783df9a9f50","Type":"ContainerStarted","Data":"d4c54c7faf30461d70b8320e58a747164ab88548b2f34db2e0dffb279ee04245"}
Jul 30 20:57:46.046619 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.046446    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:46.048011 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.047985    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx" event={"ID":"247102b0-b35c-4e25-a388-86efa0ebefd6","Type":"ContainerStarted","Data":"6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5"}
Jul 30 20:57:46.048149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.048034    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx" event={"ID":"247102b0-b35c-4e25-a388-86efa0ebefd6","Type":"ContainerStarted","Data":"f5cdefd0e5c896a19881ae56bb1be34d1bf02c5c7feb94c832200bf3114280a8"}
Jul 30 20:57:46.048149 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.048062    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:57:46.050105 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.050078    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv" event={"ID":"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290","Type":"ContainerStarted","Data":"97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa"}
Jul 30 20:57:46.050191 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.050134    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:46.050191 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.050157    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv" event={"ID":"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290","Type":"ContainerStarted","Data":"f733a1dd384b386d9290b9b75ed4f6790be8d1a20b42bf4bb539c91ef5e82d3b"}
Jul 30 20:57:46.068235 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.068201    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg" podStartSLOduration=1.068188794 podStartE2EDuration="1.068188794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:46.06795894 +0000 UTC m=+1944.528917751" watchObservedRunningTime="2026-07-30 20:57:46.068188794 +0000 UTC m=+1944.529147605"
Jul 30 20:57:46.091788 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.091745    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx" podStartSLOduration=1.091733296 podStartE2EDuration="1.091733296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:46.090354888 +0000 UTC m=+1944.551313713" watchObservedRunningTime="2026-07-30 20:57:46.091733296 +0000 UTC m=+1944.552692107"
Jul 30 20:57:46.109079 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:46.109041    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv" podStartSLOduration=1.6006257000000002 podStartE2EDuration="2.109028247s" totalImagesPullingTime="508.402547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:57:46.107094154 +0000 UTC m=+1944.568052966" watchObservedRunningTime="2026-07-30 20:57:46.109028247 +0000 UTC m=+1944.569987056"
Jul 30 20:57:52.061286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:52.061259    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:57:57.057304 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:57.057276    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:57:57.057748 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:57:57.057441    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:58:24.866103 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:24.866017    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"]
Jul 30 20:58:24.866627 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:24.866229    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv" podUID="4eeddef6-2c0b-49d6-bf4c-b9849ad6e290" containerName="maas-api" containerID="cri-o://97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa" gracePeriod=30
Jul 30 20:58:24.943547 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:24.943514    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"]
Jul 30 20:58:24.943813 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:24.943759    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg" podUID="beac278b-a580-4701-a016-e783df9a9f50" containerName="payload-processing" containerID="cri-o://bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360" gracePeriod=30
Jul 30 20:58:24.948167 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:24.948141    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"]
Jul 30 20:58:24.948410 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:24.948363    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx" podUID="247102b0-b35c-4e25-a388-86efa0ebefd6" containerName="payload-pre-processing" containerID="cri-o://6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5" gracePeriod=30
Jul 30 20:58:25.201112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.201085    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:58:25.240187 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.240162    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:58:25.244679 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.244657    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:58:25.248825 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.248797    2547 generic.go:356] "Generic (PLEG): container finished" podID="beac278b-a580-4701-a016-e783df9a9f50" containerID="bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360" exitCode=0
Jul 30 20:58:25.248955 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.248834    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"
Jul 30 20:58:25.248955 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.248885    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg" event={"ID":"beac278b-a580-4701-a016-e783df9a9f50","Type":"ContainerDied","Data":"bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360"}
Jul 30 20:58:25.248955 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.248943    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg" event={"ID":"beac278b-a580-4701-a016-e783df9a9f50","Type":"ContainerDied","Data":"d4c54c7faf30461d70b8320e58a747164ab88548b2f34db2e0dffb279ee04245"}
Jul 30 20:58:25.249116 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.248975    2547 scope.go:117] "RemoveContainer" containerID="bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360"
Jul 30 20:58:25.250216 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.250190    2547 generic.go:356] "Generic (PLEG): container finished" podID="247102b0-b35c-4e25-a388-86efa0ebefd6" containerID="6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5" exitCode=0
Jul 30 20:58:25.250368 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.250220    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx" event={"ID":"247102b0-b35c-4e25-a388-86efa0ebefd6","Type":"ContainerDied","Data":"6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5"}
Jul 30 20:58:25.250368 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.250239    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"
Jul 30 20:58:25.250368 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.250279    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx" event={"ID":"247102b0-b35c-4e25-a388-86efa0ebefd6","Type":"ContainerDied","Data":"f5cdefd0e5c896a19881ae56bb1be34d1bf02c5c7feb94c832200bf3114280a8"}
Jul 30 20:58:25.251745 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.251723    2547 generic.go:356] "Generic (PLEG): container finished" podID="4eeddef6-2c0b-49d6-bf4c-b9849ad6e290" containerID="97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa" exitCode=0
Jul 30 20:58:25.251868 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.251764    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"
Jul 30 20:58:25.251868 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.251822    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv" event={"ID":"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290","Type":"ContainerDied","Data":"97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa"}
Jul 30 20:58:25.251955 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.251863    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv" event={"ID":"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290","Type":"ContainerDied","Data":"f733a1dd384b386d9290b9b75ed4f6790be8d1a20b42bf4bb539c91ef5e82d3b"}
Jul 30 20:58:25.257745 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.257715    2547 scope.go:117] "RemoveContainer" containerID="bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360"
Jul 30 20:58:25.257985 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:58:25.257966    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360\": container with ID starting with bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360 not found: ID does not exist" containerID="bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360"
Jul 30 20:58:25.258069 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.257994    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360"} err="failed to get container status \"bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360\": rpc error: code = NotFound desc = could not find container \"bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360\": container with ID starting with bdf5a94560e8255a0433918ae2b5f543f6cd6f6c6b1478f6a815f4f2ccc1d360 not found: ID does not exist"
Jul 30 20:58:25.258069 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.258010    2547 scope.go:117] "RemoveContainer" containerID="6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5"
Jul 30 20:58:25.267754 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.267736    2547 scope.go:117] "RemoveContainer" containerID="6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5"
Jul 30 20:58:25.268050 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:58:25.268032    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5\": container with ID starting with 6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5 not found: ID does not exist" containerID="6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5"
Jul 30 20:58:25.268119 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.268057    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5"} err="failed to get container status \"6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5\": rpc error: code = NotFound desc = could not find container \"6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5\": container with ID starting with 6fcc612fd8eafcfcd4c185b6aeff98552441774881bcc1eb2d3cfbec64a5f9a5 not found: ID does not exist"
Jul 30 20:58:25.268119 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.268081    2547 scope.go:117] "RemoveContainer" containerID="97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa"
Jul 30 20:58:25.279027 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.279009    2547 scope.go:117] "RemoveContainer" containerID="97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa"
Jul 30 20:58:25.279280 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:58:25.279263    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa\": container with ID starting with 97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa not found: ID does not exist" containerID="97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa"
Jul 30 20:58:25.279333 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.279288    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa"} err="failed to get container status \"97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa\": rpc error: code = NotFound desc = could not find container \"97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa\": container with ID starting with 97d6edbf951c7f361e7c99ed416a4ffce73c2bedf282733e044f7fa3fcd6afaa not found: ID does not exist"
Jul 30 20:58:25.331136 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331113    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac278b-a580-4701-a016-e783df9a9f50-plugins-config-volume\") pod \"beac278b-a580-4701-a016-e783df9a9f50\" (UID: \"beac278b-a580-4701-a016-e783df9a9f50\") "
Jul 30 20:58:25.331222 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331150    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrpb2\" (UniqueName: \"kubernetes.io/projected/247102b0-b35c-4e25-a388-86efa0ebefd6-kube-api-access-xrpb2\") pod \"247102b0-b35c-4e25-a388-86efa0ebefd6\" (UID: \"247102b0-b35c-4e25-a388-86efa0ebefd6\") "
Jul 30 20:58:25.331222 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331177    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-openshift-service-ca\") pod \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") "
Jul 30 20:58:25.331315 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331246    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-maas-api-tls\") pod \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") "
Jul 30 20:58:25.331363 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331318    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb7cb\" (UniqueName: \"kubernetes.io/projected/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-kube-api-access-sb7cb\") pod \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\" (UID: \"4eeddef6-2c0b-49d6-bf4c-b9849ad6e290\") "
Jul 30 20:58:25.331363 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331341    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxdqr\" (UniqueName: \"kubernetes.io/projected/beac278b-a580-4701-a016-e783df9a9f50-kube-api-access-lxdqr\") pod \"beac278b-a580-4701-a016-e783df9a9f50\" (UID: \"beac278b-a580-4701-a016-e783df9a9f50\") "
Jul 30 20:58:25.331466 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331385    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/247102b0-b35c-4e25-a388-86efa0ebefd6-plugins-config-volume\") pod \"247102b0-b35c-4e25-a388-86efa0ebefd6\" (UID: \"247102b0-b35c-4e25-a388-86efa0ebefd6\") "
Jul 30 20:58:25.331554 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331527    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beac278b-a580-4701-a016-e783df9a9f50-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "beac278b-a580-4701-a016-e783df9a9f50" (UID: "beac278b-a580-4701-a016-e783df9a9f50"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:58:25.331554 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331535    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4eeddef6-2c0b-49d6-bf4c-b9849ad6e290" (UID: "4eeddef6-2c0b-49d6-bf4c-b9849ad6e290"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:58:25.331795 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331687    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac278b-a580-4701-a016-e783df9a9f50-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:58:25.331898 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331824    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:58:25.331898 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.331831    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/247102b0-b35c-4e25-a388-86efa0ebefd6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "247102b0-b35c-4e25-a388-86efa0ebefd6" (UID: "247102b0-b35c-4e25-a388-86efa0ebefd6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:58:25.333943 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.333918    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/247102b0-b35c-4e25-a388-86efa0ebefd6-kube-api-access-xrpb2" (OuterVolumeSpecName: "kube-api-access-xrpb2") pod "247102b0-b35c-4e25-a388-86efa0ebefd6" (UID: "247102b0-b35c-4e25-a388-86efa0ebefd6"). InnerVolumeSpecName "kube-api-access-xrpb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:58:25.334034 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.333960    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4eeddef6-2c0b-49d6-bf4c-b9849ad6e290" (UID: "4eeddef6-2c0b-49d6-bf4c-b9849ad6e290"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:58:25.334034 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.333957    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-kube-api-access-sb7cb" (OuterVolumeSpecName: "kube-api-access-sb7cb") pod "4eeddef6-2c0b-49d6-bf4c-b9849ad6e290" (UID: "4eeddef6-2c0b-49d6-bf4c-b9849ad6e290"). InnerVolumeSpecName "kube-api-access-sb7cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:58:25.334034 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.333991    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beac278b-a580-4701-a016-e783df9a9f50-kube-api-access-lxdqr" (OuterVolumeSpecName: "kube-api-access-lxdqr") pod "beac278b-a580-4701-a016-e783df9a9f50" (UID: "beac278b-a580-4701-a016-e783df9a9f50"). InnerVolumeSpecName "kube-api-access-lxdqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:58:25.432421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.432367    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:58:25.432421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.432388    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb7cb\" (UniqueName: \"kubernetes.io/projected/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290-kube-api-access-sb7cb\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:58:25.432421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.432399    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxdqr\" (UniqueName: \"kubernetes.io/projected/beac278b-a580-4701-a016-e783df9a9f50-kube-api-access-lxdqr\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:58:25.432421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.432407    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/247102b0-b35c-4e25-a388-86efa0ebefd6-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:58:25.432421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.432416    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrpb2\" (UniqueName: \"kubernetes.io/projected/247102b0-b35c-4e25-a388-86efa0ebefd6-kube-api-access-xrpb2\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:58:25.572486 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.572459    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"]
Jul 30 20:58:25.576663 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.576645    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-83e97498-78dd597cbb-st4sg"]
Jul 30 20:58:25.588979 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.588954    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"]
Jul 30 20:58:25.593033 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.593014    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-83e97498-585c89d855-nfkxv"]
Jul 30 20:58:25.604039 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.604010    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"]
Jul 30 20:58:25.609407 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:25.609386    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-83e97498-66dd694888-5bmhx"]
Jul 30 20:58:26.072740 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:26.072688    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="247102b0-b35c-4e25-a388-86efa0ebefd6" path="/var/lib/kubelet/pods/247102b0-b35c-4e25-a388-86efa0ebefd6/volumes"
Jul 30 20:58:26.073134 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:26.073120    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eeddef6-2c0b-49d6-bf4c-b9849ad6e290" path="/var/lib/kubelet/pods/4eeddef6-2c0b-49d6-bf4c-b9849ad6e290/volumes"
Jul 30 20:58:26.073466 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:26.073453    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beac278b-a580-4701-a016-e783df9a9f50" path="/var/lib/kubelet/pods/beac278b-a580-4701-a016-e783df9a9f50/volumes"
Jul 30 20:58:28.699486 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.699455    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"]
Jul 30 20:58:28.700054 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.700038    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247102b0-b35c-4e25-a388-86efa0ebefd6" containerName="payload-pre-processing"
Jul 30 20:58:28.700097 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.700059    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="247102b0-b35c-4e25-a388-86efa0ebefd6" containerName="payload-pre-processing"
Jul 30 20:58:28.700097 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.700083    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beac278b-a580-4701-a016-e783df9a9f50" containerName="payload-processing"
Jul 30 20:58:28.700097 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.700092    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="beac278b-a580-4701-a016-e783df9a9f50" containerName="payload-processing"
Jul 30 20:58:28.700188 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.700105    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eeddef6-2c0b-49d6-bf4c-b9849ad6e290" containerName="maas-api"
Jul 30 20:58:28.700188 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.700114    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eeddef6-2c0b-49d6-bf4c-b9849ad6e290" containerName="maas-api"
Jul 30 20:58:28.700256 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.700198    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eeddef6-2c0b-49d6-bf4c-b9849ad6e290" containerName="maas-api"
Jul 30 20:58:28.700256 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.700210    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="247102b0-b35c-4e25-a388-86efa0ebefd6" containerName="payload-pre-processing"
Jul 30 20:58:28.700256 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.700224    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="beac278b-a580-4701-a016-e783df9a9f50" containerName="payload-processing"
Jul 30 20:58:28.705071 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.705052    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:28.707740 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.707718    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-24ee2314-serving-cert\""
Jul 30 20:58:28.714148 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.714123    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"]
Jul 30 20:58:28.759926 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.759901    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe6c239a-18fb-450d-8444-3c4e152aa18a-openshift-service-ca\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:28.760032 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.759930    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfn5k\" (UniqueName: \"kubernetes.io/projected/fe6c239a-18fb-450d-8444-3c4e152aa18a-kube-api-access-xfn5k\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:28.760032 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.759959    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe6c239a-18fb-450d-8444-3c4e152aa18a-maas-api-tls\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:28.861315 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.861282    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe6c239a-18fb-450d-8444-3c4e152aa18a-openshift-service-ca\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:28.861438 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.861324    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfn5k\" (UniqueName: \"kubernetes.io/projected/fe6c239a-18fb-450d-8444-3c4e152aa18a-kube-api-access-xfn5k\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:28.861438 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.861365    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe6c239a-18fb-450d-8444-3c4e152aa18a-maas-api-tls\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:28.861574 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:58:28.861556    2547 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-serving-cert: secret "maas-api-e2e-migrate-24ee2314-serving-cert" not found
Jul 30 20:58:28.861645 ip-10-0-134-148 kubenswrapper[2547]: E0730 20:58:28.861635    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe6c239a-18fb-450d-8444-3c4e152aa18a-maas-api-tls podName:fe6c239a-18fb-450d-8444-3c4e152aa18a nodeName:}" failed. No retries permitted until 2026-07-30 20:58:29.361616486 +0000 UTC m=+1987.822575279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe6c239a-18fb-450d-8444-3c4e152aa18a-maas-api-tls") pod "maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc" (UID: "fe6c239a-18fb-450d-8444-3c4e152aa18a") : secret "maas-api-e2e-migrate-24ee2314-serving-cert" not found
Jul 30 20:58:28.861988 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.861968    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe6c239a-18fb-450d-8444-3c4e152aa18a-openshift-service-ca\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:28.871839 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:28.871821    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfn5k\" (UniqueName: \"kubernetes.io/projected/fe6c239a-18fb-450d-8444-3c4e152aa18a-kube-api-access-xfn5k\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:29.364880 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.364848    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe6c239a-18fb-450d-8444-3c4e152aa18a-maas-api-tls\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:29.367077 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.367052    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe6c239a-18fb-450d-8444-3c4e152aa18a-maas-api-tls\") pod \"maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:29.370507 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.370482    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"]
Jul 30 20:58:29.374087 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.374067    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:58:29.376929 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.376907    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-24ee2314\""
Jul 30 20:58:29.377037 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.376963    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-24ee2314-dockercfg-ddjsh\""
Jul 30 20:58:29.385220 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.385197    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"]
Jul 30 20:58:29.404360 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.404336    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"]
Jul 30 20:58:29.407615 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.407596    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:29.418556 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.418532    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"]
Jul 30 20:58:29.466067 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.466042    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g45m6\" (UniqueName: \"kubernetes.io/projected/f8d7829b-08eb-4970-997b-93b764391bf1-kube-api-access-g45m6\") pod \"payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj\" (UID: \"f8d7829b-08eb-4970-997b-93b764391bf1\") " pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:29.466166 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.466076    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9589fcf-3a63-47eb-82f2-328a16d5aa11-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw\" (UID: \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:58:29.466166 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.466154    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmlzc\" (UniqueName: \"kubernetes.io/projected/e9589fcf-3a63-47eb-82f2-328a16d5aa11-kube-api-access-rmlzc\") pod \"payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw\" (UID: \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:58:29.466238 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.466172    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8d7829b-08eb-4970-997b-93b764391bf1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj\" (UID: \"f8d7829b-08eb-4970-997b-93b764391bf1\") " pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:29.567181 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.567152    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9589fcf-3a63-47eb-82f2-328a16d5aa11-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw\" (UID: \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:58:29.567313 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.567207    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmlzc\" (UniqueName: \"kubernetes.io/projected/e9589fcf-3a63-47eb-82f2-328a16d5aa11-kube-api-access-rmlzc\") pod \"payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw\" (UID: \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:58:29.567313 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.567236    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8d7829b-08eb-4970-997b-93b764391bf1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj\" (UID: \"f8d7829b-08eb-4970-997b-93b764391bf1\") " pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:29.567313 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.567289    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g45m6\" (UniqueName: \"kubernetes.io/projected/f8d7829b-08eb-4970-997b-93b764391bf1-kube-api-access-g45m6\") pod \"payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj\" (UID: \"f8d7829b-08eb-4970-997b-93b764391bf1\") " pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:29.568057 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.568035    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9589fcf-3a63-47eb-82f2-328a16d5aa11-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw\" (UID: \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:58:29.568105 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.568035    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8d7829b-08eb-4970-997b-93b764391bf1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj\" (UID: \"f8d7829b-08eb-4970-997b-93b764391bf1\") " pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:29.576152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.576122    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmlzc\" (UniqueName: \"kubernetes.io/projected/e9589fcf-3a63-47eb-82f2-328a16d5aa11-kube-api-access-rmlzc\") pod \"payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw\" (UID: \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:58:29.576152 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.576125    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g45m6\" (UniqueName: \"kubernetes.io/projected/f8d7829b-08eb-4970-997b-93b764391bf1-kube-api-access-g45m6\") pod \"payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj\" (UID: \"f8d7829b-08eb-4970-997b-93b764391bf1\") " pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:29.615959 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.615899    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:29.686731 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.686616    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:58:29.721885 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.721752    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:29.757686 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.757230    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"]
Jul 30 20:58:29.759763 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:58:29.759680    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe6c239a_18fb_450d_8444_3c4e152aa18a.slice/crio-9180bdff0ee8982f10df3d010eb3cea916cdb441b4ed9c2af59c9f1dc27f6c22 WatchSource:0}: Error finding container 9180bdff0ee8982f10df3d010eb3cea916cdb441b4ed9c2af59c9f1dc27f6c22: Status 404 returned error can't find the container with id 9180bdff0ee8982f10df3d010eb3cea916cdb441b4ed9c2af59c9f1dc27f6c22
Jul 30 20:58:29.856627 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.856583    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"]
Jul 30 20:58:29.859324 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:58:29.859293    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9589fcf_3a63_47eb_82f2_328a16d5aa11.slice/crio-8eff85e58c66a89510a7ab08d6a4349e3aa92fb5fe4dc0d44e3b7fb9919da4d3 WatchSource:0}: Error finding container 8eff85e58c66a89510a7ab08d6a4349e3aa92fb5fe4dc0d44e3b7fb9919da4d3: Status 404 returned error can't find the container with id 8eff85e58c66a89510a7ab08d6a4349e3aa92fb5fe4dc0d44e3b7fb9919da4d3
Jul 30 20:58:29.892869 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:29.892842    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"]
Jul 30 20:58:29.894995 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:58:29.894970    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d7829b_08eb_4970_997b_93b764391bf1.slice/crio-b0ba4dfd8a478fec619d3010e74b631edd6e08a31aa71ee343ec877d7ba2e175 WatchSource:0}: Error finding container b0ba4dfd8a478fec619d3010e74b631edd6e08a31aa71ee343ec877d7ba2e175: Status 404 returned error can't find the container with id b0ba4dfd8a478fec619d3010e74b631edd6e08a31aa71ee343ec877d7ba2e175
Jul 30 20:58:30.282884 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.282853    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw" event={"ID":"e9589fcf-3a63-47eb-82f2-328a16d5aa11","Type":"ContainerStarted","Data":"9adbd3f4db74b5e529b0d11deff519a88b9cbf7880c47eaee04e89d9fafb74c9"}
Jul 30 20:58:30.283014 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.282910    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw" event={"ID":"e9589fcf-3a63-47eb-82f2-328a16d5aa11","Type":"ContainerStarted","Data":"8eff85e58c66a89510a7ab08d6a4349e3aa92fb5fe4dc0d44e3b7fb9919da4d3"}
Jul 30 20:58:30.283014 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.282936    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:58:30.284433 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.284401    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc" event={"ID":"fe6c239a-18fb-450d-8444-3c4e152aa18a","Type":"ContainerStarted","Data":"bf694293485306c159c8b180eff68b589fe54a18738d56dd18ef016e704a882c"}
Jul 30 20:58:30.284542 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.284459    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:30.284542 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.284484    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc" event={"ID":"fe6c239a-18fb-450d-8444-3c4e152aa18a","Type":"ContainerStarted","Data":"9180bdff0ee8982f10df3d010eb3cea916cdb441b4ed9c2af59c9f1dc27f6c22"}
Jul 30 20:58:30.285771 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.285751    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj" event={"ID":"f8d7829b-08eb-4970-997b-93b764391bf1","Type":"ContainerStarted","Data":"dada30c89bf91e6b08e16a9627d161ca14fa66d91c76a540a6b57a1cf5c87e77"}
Jul 30 20:58:30.285867 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.285786    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj" event={"ID":"f8d7829b-08eb-4970-997b-93b764391bf1","Type":"ContainerStarted","Data":"b0ba4dfd8a478fec619d3010e74b631edd6e08a31aa71ee343ec877d7ba2e175"}
Jul 30 20:58:30.285908 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.285872    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:30.310909 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.310868    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw" podStartSLOduration=1.310852919 podStartE2EDuration="1.310852919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:30.308572677 +0000 UTC m=+1988.769531492" watchObservedRunningTime="2026-07-30 20:58:30.310852919 +0000 UTC m=+1988.771811781"
Jul 30 20:58:30.326984 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.326927    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc" podStartSLOduration=1.893508456 podStartE2EDuration="2.32690915s" totalImagesPullingTime="433.400694ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:30.326390077 +0000 UTC m=+1988.787348889" watchObservedRunningTime="2026-07-30 20:58:30.32690915 +0000 UTC m=+1988.787867961"
Jul 30 20:58:30.344445 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:30.344404    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj" podStartSLOduration=1.344390751 podStartE2EDuration="1.344390751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:58:30.342255065 +0000 UTC m=+1988.803213877" watchObservedRunningTime="2026-07-30 20:58:30.344390751 +0000 UTC m=+1988.805349602"
Jul 30 20:58:36.295387 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:36.295359    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:58:41.292848 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:41.292812    2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw" podUID="e9589fcf-3a63-47eb-82f2-328a16d5aa11" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.49:9004: connect: connection refused"
Jul 30 20:58:41.293317 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:41.293012    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:58:51.291283 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:58:51.291257    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:59:00.210446 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.210411    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"]
Jul 30 20:59:00.212896 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.210628    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc" podUID="fe6c239a-18fb-450d-8444-3c4e152aa18a" containerName="maas-api" containerID="cri-o://bf694293485306c159c8b180eff68b589fe54a18738d56dd18ef016e704a882c" gracePeriod=30
Jul 30 20:59:00.276714 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.276676    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"]
Jul 30 20:59:00.276927 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.276903    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj" podUID="f8d7829b-08eb-4970-997b-93b764391bf1" containerName="payload-processing" containerID="cri-o://dada30c89bf91e6b08e16a9627d161ca14fa66d91c76a540a6b57a1cf5c87e77" gracePeriod=30
Jul 30 20:59:00.291014 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.290981    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"]
Jul 30 20:59:00.291256 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.291233    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw" podUID="e9589fcf-3a63-47eb-82f2-328a16d5aa11" containerName="payload-pre-processing" containerID="cri-o://9adbd3f4db74b5e529b0d11deff519a88b9cbf7880c47eaee04e89d9fafb74c9" gracePeriod=30
Jul 30 20:59:00.441369 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.441302    2547 generic.go:356] "Generic (PLEG): container finished" podID="fe6c239a-18fb-450d-8444-3c4e152aa18a" containerID="bf694293485306c159c8b180eff68b589fe54a18738d56dd18ef016e704a882c" exitCode=0
Jul 30 20:59:00.441564 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.441467    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc" event={"ID":"fe6c239a-18fb-450d-8444-3c4e152aa18a","Type":"ContainerDied","Data":"bf694293485306c159c8b180eff68b589fe54a18738d56dd18ef016e704a882c"}
Jul 30 20:59:00.443450 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.443416    2547 generic.go:356] "Generic (PLEG): container finished" podID="f8d7829b-08eb-4970-997b-93b764391bf1" containerID="dada30c89bf91e6b08e16a9627d161ca14fa66d91c76a540a6b57a1cf5c87e77" exitCode=0
Jul 30 20:59:00.443576 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.443503    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj" event={"ID":"f8d7829b-08eb-4970-997b-93b764391bf1","Type":"ContainerDied","Data":"dada30c89bf91e6b08e16a9627d161ca14fa66d91c76a540a6b57a1cf5c87e77"}
Jul 30 20:59:00.445388 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.445364    2547 generic.go:356] "Generic (PLEG): container finished" podID="e9589fcf-3a63-47eb-82f2-328a16d5aa11" containerID="9adbd3f4db74b5e529b0d11deff519a88b9cbf7880c47eaee04e89d9fafb74c9" exitCode=0
Jul 30 20:59:00.445620 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.445599    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw" event={"ID":"e9589fcf-3a63-47eb-82f2-328a16d5aa11","Type":"ContainerDied","Data":"9adbd3f4db74b5e529b0d11deff519a88b9cbf7880c47eaee04e89d9fafb74c9"}
Jul 30 20:59:00.557816 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.557787    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:59:00.564493 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.563860    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:59:00.600922 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.600890    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:59:00.719056 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.718985    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8d7829b-08eb-4970-997b-93b764391bf1-plugins-config-volume\") pod \"f8d7829b-08eb-4970-997b-93b764391bf1\" (UID: \"f8d7829b-08eb-4970-997b-93b764391bf1\") "
Jul 30 20:59:00.719056 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719041    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe6c239a-18fb-450d-8444-3c4e152aa18a-maas-api-tls\") pod \"fe6c239a-18fb-450d-8444-3c4e152aa18a\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") "
Jul 30 20:59:00.719222 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719126    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe6c239a-18fb-450d-8444-3c4e152aa18a-openshift-service-ca\") pod \"fe6c239a-18fb-450d-8444-3c4e152aa18a\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") "
Jul 30 20:59:00.719222 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719158    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g45m6\" (UniqueName: \"kubernetes.io/projected/f8d7829b-08eb-4970-997b-93b764391bf1-kube-api-access-g45m6\") pod \"f8d7829b-08eb-4970-997b-93b764391bf1\" (UID: \"f8d7829b-08eb-4970-997b-93b764391bf1\") "
Jul 30 20:59:00.719222 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719194    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmlzc\" (UniqueName: \"kubernetes.io/projected/e9589fcf-3a63-47eb-82f2-328a16d5aa11-kube-api-access-rmlzc\") pod \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\" (UID: \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\") "
Jul 30 20:59:00.719376 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719266    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfn5k\" (UniqueName: \"kubernetes.io/projected/fe6c239a-18fb-450d-8444-3c4e152aa18a-kube-api-access-xfn5k\") pod \"fe6c239a-18fb-450d-8444-3c4e152aa18a\" (UID: \"fe6c239a-18fb-450d-8444-3c4e152aa18a\") "
Jul 30 20:59:00.719376 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719299    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9589fcf-3a63-47eb-82f2-328a16d5aa11-plugins-config-volume\") pod \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\" (UID: \"e9589fcf-3a63-47eb-82f2-328a16d5aa11\") "
Jul 30 20:59:00.719480 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719398    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8d7829b-08eb-4970-997b-93b764391bf1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8d7829b-08eb-4970-997b-93b764391bf1" (UID: "f8d7829b-08eb-4970-997b-93b764391bf1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:59:00.719480 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719461    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe6c239a-18fb-450d-8444-3c4e152aa18a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe6c239a-18fb-450d-8444-3c4e152aa18a" (UID: "fe6c239a-18fb-450d-8444-3c4e152aa18a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:59:00.719853 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719820    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9589fcf-3a63-47eb-82f2-328a16d5aa11-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e9589fcf-3a63-47eb-82f2-328a16d5aa11" (UID: "e9589fcf-3a63-47eb-82f2-328a16d5aa11"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:59:00.720012 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.719864    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8d7829b-08eb-4970-997b-93b764391bf1-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:00.720298 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.720276    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe6c239a-18fb-450d-8444-3c4e152aa18a-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:00.721456 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.721428    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe6c239a-18fb-450d-8444-3c4e152aa18a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe6c239a-18fb-450d-8444-3c4e152aa18a" (UID: "fe6c239a-18fb-450d-8444-3c4e152aa18a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:59:00.721456 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.721442    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8d7829b-08eb-4970-997b-93b764391bf1-kube-api-access-g45m6" (OuterVolumeSpecName: "kube-api-access-g45m6") pod "f8d7829b-08eb-4970-997b-93b764391bf1" (UID: "f8d7829b-08eb-4970-997b-93b764391bf1"). InnerVolumeSpecName "kube-api-access-g45m6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:59:00.721572 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.721501    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9589fcf-3a63-47eb-82f2-328a16d5aa11-kube-api-access-rmlzc" (OuterVolumeSpecName: "kube-api-access-rmlzc") pod "e9589fcf-3a63-47eb-82f2-328a16d5aa11" (UID: "e9589fcf-3a63-47eb-82f2-328a16d5aa11"). InnerVolumeSpecName "kube-api-access-rmlzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:59:00.721572 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.721514    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe6c239a-18fb-450d-8444-3c4e152aa18a-kube-api-access-xfn5k" (OuterVolumeSpecName: "kube-api-access-xfn5k") pod "fe6c239a-18fb-450d-8444-3c4e152aa18a" (UID: "fe6c239a-18fb-450d-8444-3c4e152aa18a"). InnerVolumeSpecName "kube-api-access-xfn5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:59:00.821318 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.821296    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g45m6\" (UniqueName: \"kubernetes.io/projected/f8d7829b-08eb-4970-997b-93b764391bf1-kube-api-access-g45m6\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:00.821318 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.821319    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmlzc\" (UniqueName: \"kubernetes.io/projected/e9589fcf-3a63-47eb-82f2-328a16d5aa11-kube-api-access-rmlzc\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:00.821455 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.821328    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfn5k\" (UniqueName: \"kubernetes.io/projected/fe6c239a-18fb-450d-8444-3c4e152aa18a-kube-api-access-xfn5k\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:00.821455 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.821337    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9589fcf-3a63-47eb-82f2-328a16d5aa11-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:00.821455 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:00.821345    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe6c239a-18fb-450d-8444-3c4e152aa18a-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:01.452349 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.452316    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"
Jul 30 20:59:01.452797 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.452353    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc" event={"ID":"fe6c239a-18fb-450d-8444-3c4e152aa18a","Type":"ContainerDied","Data":"9180bdff0ee8982f10df3d010eb3cea916cdb441b4ed9c2af59c9f1dc27f6c22"}
Jul 30 20:59:01.452797 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.452420    2547 scope.go:117] "RemoveContainer" containerID="bf694293485306c159c8b180eff68b589fe54a18738d56dd18ef016e704a882c"
Jul 30 20:59:01.454189 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.454168    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"
Jul 30 20:59:01.454334 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.454168    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj" event={"ID":"f8d7829b-08eb-4970-997b-93b764391bf1","Type":"ContainerDied","Data":"b0ba4dfd8a478fec619d3010e74b631edd6e08a31aa71ee343ec877d7ba2e175"}
Jul 30 20:59:01.455756 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.455733    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"
Jul 30 20:59:01.455887 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.455736    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw" event={"ID":"e9589fcf-3a63-47eb-82f2-328a16d5aa11","Type":"ContainerDied","Data":"8eff85e58c66a89510a7ab08d6a4349e3aa92fb5fe4dc0d44e3b7fb9919da4d3"}
Jul 30 20:59:01.461744 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.461716    2547 scope.go:117] "RemoveContainer" containerID="dada30c89bf91e6b08e16a9627d161ca14fa66d91c76a540a6b57a1cf5c87e77"
Jul 30 20:59:01.472564 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.472548    2547 scope.go:117] "RemoveContainer" containerID="9adbd3f4db74b5e529b0d11deff519a88b9cbf7880c47eaee04e89d9fafb74c9"
Jul 30 20:59:01.485858 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.485834    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"]
Jul 30 20:59:01.489421 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.489400    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-24ee2314-6dfdb47c4c-t8szc"]
Jul 30 20:59:01.501473 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.501451    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"]
Jul 30 20:59:01.505335 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.505306    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-24ee2314-6df999ccb-ksfrw"]
Jul 30 20:59:01.518034 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.518012    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"]
Jul 30 20:59:01.526088 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:01.526056    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-24ee2314-74cbb67549-bq6rj"]
Jul 30 20:59:02.072235 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:02.072206    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9589fcf-3a63-47eb-82f2-328a16d5aa11" path="/var/lib/kubelet/pods/e9589fcf-3a63-47eb-82f2-328a16d5aa11/volumes"
Jul 30 20:59:02.072552 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:02.072531    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8d7829b-08eb-4970-997b-93b764391bf1" path="/var/lib/kubelet/pods/f8d7829b-08eb-4970-997b-93b764391bf1/volumes"
Jul 30 20:59:02.072885 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:02.072872    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe6c239a-18fb-450d-8444-3c4e152aa18a" path="/var/lib/kubelet/pods/fe6c239a-18fb-450d-8444-3c4e152aa18a/volumes"
Jul 30 20:59:04.717144 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717110    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"]
Jul 30 20:59:04.717499 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717474    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe6c239a-18fb-450d-8444-3c4e152aa18a" containerName="maas-api"
Jul 30 20:59:04.717499 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717485    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe6c239a-18fb-450d-8444-3c4e152aa18a" containerName="maas-api"
Jul 30 20:59:04.717574 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717505    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9589fcf-3a63-47eb-82f2-328a16d5aa11" containerName="payload-pre-processing"
Jul 30 20:59:04.717574 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717512    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9589fcf-3a63-47eb-82f2-328a16d5aa11" containerName="payload-pre-processing"
Jul 30 20:59:04.717574 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717520    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8d7829b-08eb-4970-997b-93b764391bf1" containerName="payload-processing"
Jul 30 20:59:04.717574 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717526    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8d7829b-08eb-4970-997b-93b764391bf1" containerName="payload-processing"
Jul 30 20:59:04.717694 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717591    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9589fcf-3a63-47eb-82f2-328a16d5aa11" containerName="payload-pre-processing"
Jul 30 20:59:04.717694 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717600    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe6c239a-18fb-450d-8444-3c4e152aa18a" containerName="maas-api"
Jul 30 20:59:04.717694 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.717607    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8d7829b-08eb-4970-997b-93b764391bf1" containerName="payload-processing"
Jul 30 20:59:04.722442 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.722423    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:04.725331 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.725313    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-be8f5bad-serving-cert\""
Jul 30 20:59:04.732350 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.732328    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"]
Jul 30 20:59:04.852035 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.852009    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf5446f-6c27-46a0-9716-236cec288e98-maas-api-tls\") pod \"maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:04.852161 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.852069    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf5446f-6c27-46a0-9716-236cec288e98-openshift-service-ca\") pod \"maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:04.852161 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.852126    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbql5\" (UniqueName: \"kubernetes.io/projected/ecf5446f-6c27-46a0-9716-236cec288e98-kube-api-access-pbql5\") pod \"maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:04.952989 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.952961    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf5446f-6c27-46a0-9716-236cec288e98-maas-api-tls\") pod \"maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:04.953108 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.953022    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf5446f-6c27-46a0-9716-236cec288e98-openshift-service-ca\") pod \"maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:04.953108 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.953047    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbql5\" (UniqueName: \"kubernetes.io/projected/ecf5446f-6c27-46a0-9716-236cec288e98-kube-api-access-pbql5\") pod \"maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:04.953688 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.953671    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf5446f-6c27-46a0-9716-236cec288e98-openshift-service-ca\") pod \"maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:04.955559 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.955540    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf5446f-6c27-46a0-9716-236cec288e98-maas-api-tls\") pod \"maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:04.961665 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:04.961644    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbql5\" (UniqueName: \"kubernetes.io/projected/ecf5446f-6c27-46a0-9716-236cec288e98-kube-api-access-pbql5\") pod \"maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:05.034213 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.034158    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:05.187331 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.187298    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"]
Jul 30 20:59:05.201090 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:59:05.201063    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf5446f_6c27_46a0_9716_236cec288e98.slice/crio-ad4629f92d31c29137d738aa35856a245360d828e16ec0bed481dfb697ceac0c WatchSource:0}: Error finding container ad4629f92d31c29137d738aa35856a245360d828e16ec0bed481dfb697ceac0c: Status 404 returned error can't find the container with id ad4629f92d31c29137d738aa35856a245360d828e16ec0bed481dfb697ceac0c
Jul 30 20:59:05.478544 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.478515    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6" event={"ID":"ecf5446f-6c27-46a0-9716-236cec288e98","Type":"ContainerStarted","Data":"ad4629f92d31c29137d738aa35856a245360d828e16ec0bed481dfb697ceac0c"}
Jul 30 20:59:05.573134 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.573103    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"]
Jul 30 20:59:05.576627 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.576608    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:05.579729 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.579679    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-be8f5bad-dockercfg-md8z4\""
Jul 30 20:59:05.579841 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.579827    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-be8f5bad\""
Jul 30 20:59:05.585503 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.585486    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"]
Jul 30 20:59:05.592560 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.592536    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"]
Jul 30 20:59:05.592674 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.592649    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:05.603196 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.603172    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"]
Jul 30 20:59:05.761319 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.761250    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvlg6\" (UniqueName: \"kubernetes.io/projected/575f1940-21ef-4b88-ad44-164c9272efdd-kube-api-access-gvlg6\") pod \"payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf\" (UID: \"575f1940-21ef-4b88-ad44-164c9272efdd\") " pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:05.761319 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.761285    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prxls\" (UniqueName: \"kubernetes.io/projected/31d27217-e82d-4bf4-8b36-62032f67517d-kube-api-access-prxls\") pod \"payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc\" (UID: \"31d27217-e82d-4bf4-8b36-62032f67517d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:05.761663 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.761396    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31d27217-e82d-4bf4-8b36-62032f67517d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc\" (UID: \"31d27217-e82d-4bf4-8b36-62032f67517d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:05.761663 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.761413    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/575f1940-21ef-4b88-ad44-164c9272efdd-plugins-config-volume\") pod \"payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf\" (UID: \"575f1940-21ef-4b88-ad44-164c9272efdd\") " pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:05.861815 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.861772    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31d27217-e82d-4bf4-8b36-62032f67517d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc\" (UID: \"31d27217-e82d-4bf4-8b36-62032f67517d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:05.861815 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.861817    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/575f1940-21ef-4b88-ad44-164c9272efdd-plugins-config-volume\") pod \"payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf\" (UID: \"575f1940-21ef-4b88-ad44-164c9272efdd\") " pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:05.862008 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.861879    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvlg6\" (UniqueName: \"kubernetes.io/projected/575f1940-21ef-4b88-ad44-164c9272efdd-kube-api-access-gvlg6\") pod \"payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf\" (UID: \"575f1940-21ef-4b88-ad44-164c9272efdd\") " pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:05.862008 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.861897    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prxls\" (UniqueName: \"kubernetes.io/projected/31d27217-e82d-4bf4-8b36-62032f67517d-kube-api-access-prxls\") pod \"payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc\" (UID: \"31d27217-e82d-4bf4-8b36-62032f67517d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:05.862605 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.862578    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31d27217-e82d-4bf4-8b36-62032f67517d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc\" (UID: \"31d27217-e82d-4bf4-8b36-62032f67517d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:05.862727 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.862630    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/575f1940-21ef-4b88-ad44-164c9272efdd-plugins-config-volume\") pod \"payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf\" (UID: \"575f1940-21ef-4b88-ad44-164c9272efdd\") " pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:05.870781 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.870744    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvlg6\" (UniqueName: \"kubernetes.io/projected/575f1940-21ef-4b88-ad44-164c9272efdd-kube-api-access-gvlg6\") pod \"payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf\" (UID: \"575f1940-21ef-4b88-ad44-164c9272efdd\") " pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:05.870937 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.870919    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prxls\" (UniqueName: \"kubernetes.io/projected/31d27217-e82d-4bf4-8b36-62032f67517d-kube-api-access-prxls\") pod \"payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc\" (UID: \"31d27217-e82d-4bf4-8b36-62032f67517d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:05.888580 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.888557    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:05.922303 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:05.922278    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:06.249886 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.244221    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"]
Jul 30 20:59:06.285052 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.285027    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"]
Jul 30 20:59:06.289824 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:59:06.289797    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d27217_e82d_4bf4_8b36_62032f67517d.slice/crio-90f547ab8e403f7e8af04c5db400bd52f02e9b6be044b51b14e2de7941866822 WatchSource:0}: Error finding container 90f547ab8e403f7e8af04c5db400bd52f02e9b6be044b51b14e2de7941866822: Status 404 returned error can't find the container with id 90f547ab8e403f7e8af04c5db400bd52f02e9b6be044b51b14e2de7941866822
Jul 30 20:59:06.484919 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.484883    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6" event={"ID":"ecf5446f-6c27-46a0-9716-236cec288e98","Type":"ContainerStarted","Data":"70a4ac266af2abfc561557ba6b89c477393c8519baaa3fc6b948529f4b08f4d4"}
Jul 30 20:59:06.485088 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.484986    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:06.489324 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.489298    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf" event={"ID":"575f1940-21ef-4b88-ad44-164c9272efdd","Type":"ContainerStarted","Data":"c1aa0c5e535c2c1edb74b0f94da03b46dee8441b17e1fd899c40b07aad9fb3ea"}
Jul 30 20:59:06.489452 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.489342    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf" event={"ID":"575f1940-21ef-4b88-ad44-164c9272efdd","Type":"ContainerStarted","Data":"af6b1e509055d9fa00818d037b9b4534e0844a73a57e20959c83f9536af96f66"}
Jul 30 20:59:06.489452 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.489374    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:06.491010 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.490987    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc" event={"ID":"31d27217-e82d-4bf4-8b36-62032f67517d","Type":"ContainerStarted","Data":"ea59a627343fdeccad2927c28e715cb52e81210d65a14303afce230ed565a25e"}
Jul 30 20:59:06.491097 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.491025    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc" event={"ID":"31d27217-e82d-4bf4-8b36-62032f67517d","Type":"ContainerStarted","Data":"90f547ab8e403f7e8af04c5db400bd52f02e9b6be044b51b14e2de7941866822"}
Jul 30 20:59:06.491097 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.491081    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:06.504806 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.504643    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6" podStartSLOduration=2.135550333 podStartE2EDuration="2.504627078s" totalImagesPullingTime="369.076745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:06.504267732 +0000 UTC m=+2024.965226555" watchObservedRunningTime="2026-07-30 20:59:06.504627078 +0000 UTC m=+2024.965585868"
Jul 30 20:59:06.523383 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.523342    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf" podStartSLOduration=1.5233301959999999 podStartE2EDuration="1.523330196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:06.521955511 +0000 UTC m=+2024.982914333" watchObservedRunningTime="2026-07-30 20:59:06.523330196 +0000 UTC m=+2024.984289006"
Jul 30 20:59:06.548078 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:06.548001    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc" podStartSLOduration=1.547989357 podStartE2EDuration="1.547989357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:06.54683566 +0000 UTC m=+2025.007794471" watchObservedRunningTime="2026-07-30 20:59:06.547989357 +0000 UTC m=+2025.008948168"
Jul 30 20:59:12.503443 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:12.503413    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:17.498424 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:17.498395    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:17.498821 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:17.498459    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:35.402398 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.402364    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"]
Jul 30 20:59:35.403130 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.403096    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6" podUID="ecf5446f-6c27-46a0-9716-236cec288e98" containerName="maas-api" containerID="cri-o://70a4ac266af2abfc561557ba6b89c477393c8519baaa3fc6b948529f4b08f4d4" gracePeriod=30
Jul 30 20:59:35.479589 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.479556    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"]
Jul 30 20:59:35.479835 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.479808    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf" podUID="575f1940-21ef-4b88-ad44-164c9272efdd" containerName="payload-processing" containerID="cri-o://c1aa0c5e535c2c1edb74b0f94da03b46dee8441b17e1fd899c40b07aad9fb3ea" gracePeriod=30
Jul 30 20:59:35.486459 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.486095    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"]
Jul 30 20:59:35.486459 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.486294    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc" podUID="31d27217-e82d-4bf4-8b36-62032f67517d" containerName="payload-pre-processing" containerID="cri-o://ea59a627343fdeccad2927c28e715cb52e81210d65a14303afce230ed565a25e" gracePeriod=30
Jul 30 20:59:35.638314 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.638277    2547 generic.go:356] "Generic (PLEG): container finished" podID="31d27217-e82d-4bf4-8b36-62032f67517d" containerID="ea59a627343fdeccad2927c28e715cb52e81210d65a14303afce230ed565a25e" exitCode=0
Jul 30 20:59:35.638506 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.638337    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc" event={"ID":"31d27217-e82d-4bf4-8b36-62032f67517d","Type":"ContainerDied","Data":"ea59a627343fdeccad2927c28e715cb52e81210d65a14303afce230ed565a25e"}
Jul 30 20:59:35.640085 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.640054    2547 generic.go:356] "Generic (PLEG): container finished" podID="ecf5446f-6c27-46a0-9716-236cec288e98" containerID="70a4ac266af2abfc561557ba6b89c477393c8519baaa3fc6b948529f4b08f4d4" exitCode=0
Jul 30 20:59:35.640220 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.640135    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6" event={"ID":"ecf5446f-6c27-46a0-9716-236cec288e98","Type":"ContainerDied","Data":"70a4ac266af2abfc561557ba6b89c477393c8519baaa3fc6b948529f4b08f4d4"}
Jul 30 20:59:35.641988 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.641960    2547 generic.go:356] "Generic (PLEG): container finished" podID="575f1940-21ef-4b88-ad44-164c9272efdd" containerID="c1aa0c5e535c2c1edb74b0f94da03b46dee8441b17e1fd899c40b07aad9fb3ea" exitCode=0
Jul 30 20:59:35.642086 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.642002    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf" event={"ID":"575f1940-21ef-4b88-ad44-164c9272efdd","Type":"ContainerDied","Data":"c1aa0c5e535c2c1edb74b0f94da03b46dee8441b17e1fd899c40b07aad9fb3ea"}
Jul 30 20:59:35.772537 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.772511    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:35.775286 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.775266    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:35.785495 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.785477    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:35.894468 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.894444    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvlg6\" (UniqueName: \"kubernetes.io/projected/575f1940-21ef-4b88-ad44-164c9272efdd-kube-api-access-gvlg6\") pod \"575f1940-21ef-4b88-ad44-164c9272efdd\" (UID: \"575f1940-21ef-4b88-ad44-164c9272efdd\") "
Jul 30 20:59:35.894634 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.894486    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/575f1940-21ef-4b88-ad44-164c9272efdd-plugins-config-volume\") pod \"575f1940-21ef-4b88-ad44-164c9272efdd\" (UID: \"575f1940-21ef-4b88-ad44-164c9272efdd\") "
Jul 30 20:59:35.894634 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.894549    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf5446f-6c27-46a0-9716-236cec288e98-maas-api-tls\") pod \"ecf5446f-6c27-46a0-9716-236cec288e98\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") "
Jul 30 20:59:35.894634 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.894576    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prxls\" (UniqueName: \"kubernetes.io/projected/31d27217-e82d-4bf4-8b36-62032f67517d-kube-api-access-prxls\") pod \"31d27217-e82d-4bf4-8b36-62032f67517d\" (UID: \"31d27217-e82d-4bf4-8b36-62032f67517d\") "
Jul 30 20:59:35.894634 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.894607    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31d27217-e82d-4bf4-8b36-62032f67517d-plugins-config-volume\") pod \"31d27217-e82d-4bf4-8b36-62032f67517d\" (UID: \"31d27217-e82d-4bf4-8b36-62032f67517d\") "
Jul 30 20:59:35.894876 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.894857    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf5446f-6c27-46a0-9716-236cec288e98-openshift-service-ca\") pod \"ecf5446f-6c27-46a0-9716-236cec288e98\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") "
Jul 30 20:59:35.895020 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.894920    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/575f1940-21ef-4b88-ad44-164c9272efdd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "575f1940-21ef-4b88-ad44-164c9272efdd" (UID: "575f1940-21ef-4b88-ad44-164c9272efdd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:59:35.895020 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.894935    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbql5\" (UniqueName: \"kubernetes.io/projected/ecf5446f-6c27-46a0-9716-236cec288e98-kube-api-access-pbql5\") pod \"ecf5446f-6c27-46a0-9716-236cec288e98\" (UID: \"ecf5446f-6c27-46a0-9716-236cec288e98\") "
Jul 30 20:59:35.895020 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.894976    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d27217-e82d-4bf4-8b36-62032f67517d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31d27217-e82d-4bf4-8b36-62032f67517d" (UID: "31d27217-e82d-4bf4-8b36-62032f67517d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:59:35.895317 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.895243    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecf5446f-6c27-46a0-9716-236cec288e98-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ecf5446f-6c27-46a0-9716-236cec288e98" (UID: "ecf5446f-6c27-46a0-9716-236cec288e98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:59:35.895418 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.895399    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/575f1940-21ef-4b88-ad44-164c9272efdd-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:35.895489 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.895426    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31d27217-e82d-4bf4-8b36-62032f67517d-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:35.895489 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.895441    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf5446f-6c27-46a0-9716-236cec288e98-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:35.897032 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.897008    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf5446f-6c27-46a0-9716-236cec288e98-kube-api-access-pbql5" (OuterVolumeSpecName: "kube-api-access-pbql5") pod "ecf5446f-6c27-46a0-9716-236cec288e98" (UID: "ecf5446f-6c27-46a0-9716-236cec288e98"). InnerVolumeSpecName "kube-api-access-pbql5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:59:35.897170 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.897154    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/575f1940-21ef-4b88-ad44-164c9272efdd-kube-api-access-gvlg6" (OuterVolumeSpecName: "kube-api-access-gvlg6") pod "575f1940-21ef-4b88-ad44-164c9272efdd" (UID: "575f1940-21ef-4b88-ad44-164c9272efdd"). InnerVolumeSpecName "kube-api-access-gvlg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:59:35.897270 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.897251    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecf5446f-6c27-46a0-9716-236cec288e98-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ecf5446f-6c27-46a0-9716-236cec288e98" (UID: "ecf5446f-6c27-46a0-9716-236cec288e98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:59:35.897322 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.897291    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d27217-e82d-4bf4-8b36-62032f67517d-kube-api-access-prxls" (OuterVolumeSpecName: "kube-api-access-prxls") pod "31d27217-e82d-4bf4-8b36-62032f67517d" (UID: "31d27217-e82d-4bf4-8b36-62032f67517d"). InnerVolumeSpecName "kube-api-access-prxls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:59:35.996715 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.996639    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf5446f-6c27-46a0-9716-236cec288e98-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:35.996715 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.996661    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prxls\" (UniqueName: \"kubernetes.io/projected/31d27217-e82d-4bf4-8b36-62032f67517d-kube-api-access-prxls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:35.996715 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.996674    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbql5\" (UniqueName: \"kubernetes.io/projected/ecf5446f-6c27-46a0-9716-236cec288e98-kube-api-access-pbql5\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:35.996715 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:35.996686    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvlg6\" (UniqueName: \"kubernetes.io/projected/575f1940-21ef-4b88-ad44-164c9272efdd-kube-api-access-gvlg6\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 20:59:36.648169 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.648139    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"
Jul 30 20:59:36.648577 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.648166    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc" event={"ID":"31d27217-e82d-4bf4-8b36-62032f67517d","Type":"ContainerDied","Data":"90f547ab8e403f7e8af04c5db400bd52f02e9b6be044b51b14e2de7941866822"}
Jul 30 20:59:36.648577 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.648239    2547 scope.go:117] "RemoveContainer" containerID="ea59a627343fdeccad2927c28e715cb52e81210d65a14303afce230ed565a25e"
Jul 30 20:59:36.649574 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.649557    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"
Jul 30 20:59:36.649648 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.649556    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6" event={"ID":"ecf5446f-6c27-46a0-9716-236cec288e98","Type":"ContainerDied","Data":"ad4629f92d31c29137d738aa35856a245360d828e16ec0bed481dfb697ceac0c"}
Jul 30 20:59:36.651100 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.651076    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf" event={"ID":"575f1940-21ef-4b88-ad44-164c9272efdd","Type":"ContainerDied","Data":"af6b1e509055d9fa00818d037b9b4534e0844a73a57e20959c83f9536af96f66"}
Jul 30 20:59:36.651232 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.651118    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"
Jul 30 20:59:36.657991 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.657974    2547 scope.go:117] "RemoveContainer" containerID="70a4ac266af2abfc561557ba6b89c477393c8519baaa3fc6b948529f4b08f4d4"
Jul 30 20:59:36.665467 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.665451    2547 scope.go:117] "RemoveContainer" containerID="c1aa0c5e535c2c1edb74b0f94da03b46dee8441b17e1fd899c40b07aad9fb3ea"
Jul 30 20:59:36.672562 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.672543    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"]
Jul 30 20:59:36.678532 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.678510    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be8f5bad-6d7845c55d-gpstf"]
Jul 30 20:59:36.689712 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.689684    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"]
Jul 30 20:59:36.694204 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.694184    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be8f5bad-654f66f944-s7dgc"]
Jul 30 20:59:36.705081 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.705056    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"]
Jul 30 20:59:36.709447 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:36.709428    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be8f5bad-778c58795f-2fjn6"]
Jul 30 20:59:38.073266 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:38.073235    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31d27217-e82d-4bf4-8b36-62032f67517d" path="/var/lib/kubelet/pods/31d27217-e82d-4bf4-8b36-62032f67517d/volumes"
Jul 30 20:59:38.073602 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:38.073587    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="575f1940-21ef-4b88-ad44-164c9272efdd" path="/var/lib/kubelet/pods/575f1940-21ef-4b88-ad44-164c9272efdd/volumes"
Jul 30 20:59:38.073959 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:38.073945    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecf5446f-6c27-46a0-9716-236cec288e98" path="/var/lib/kubelet/pods/ecf5446f-6c27-46a0-9716-236cec288e98/volumes"
Jul 30 20:59:40.926596 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.926567    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"]
Jul 30 20:59:40.927051 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.927035    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575f1940-21ef-4b88-ad44-164c9272efdd" containerName="payload-processing"
Jul 30 20:59:40.927112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.927054    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="575f1940-21ef-4b88-ad44-164c9272efdd" containerName="payload-processing"
Jul 30 20:59:40.927112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.927073    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d27217-e82d-4bf4-8b36-62032f67517d" containerName="payload-pre-processing"
Jul 30 20:59:40.927112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.927079    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d27217-e82d-4bf4-8b36-62032f67517d" containerName="payload-pre-processing"
Jul 30 20:59:40.927112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.927090    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecf5446f-6c27-46a0-9716-236cec288e98" containerName="maas-api"
Jul 30 20:59:40.927112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.927095    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecf5446f-6c27-46a0-9716-236cec288e98" containerName="maas-api"
Jul 30 20:59:40.927259 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.927153    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecf5446f-6c27-46a0-9716-236cec288e98" containerName="maas-api"
Jul 30 20:59:40.927259 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.927162    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="31d27217-e82d-4bf4-8b36-62032f67517d" containerName="payload-pre-processing"
Jul 30 20:59:40.927259 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.927169    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="575f1940-21ef-4b88-ad44-164c9272efdd" containerName="payload-processing"
Jul 30 20:59:40.932096 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.932074    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:40.935040 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.935018    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-052492e6-serving-cert\""
Jul 30 20:59:40.940753 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:40.940629    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"]
Jul 30 20:59:41.035572 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.035538    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-maas-api-tls\") pod \"maas-api-e2e-derive-052492e6-7757568586-2k45z\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.035752 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.035581    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rbj2\" (UniqueName: \"kubernetes.io/projected/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-kube-api-access-5rbj2\") pod \"maas-api-e2e-derive-052492e6-7757568586-2k45z\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.035752 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.035687    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-openshift-service-ca\") pod \"maas-api-e2e-derive-052492e6-7757568586-2k45z\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.136998 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.136968    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-maas-api-tls\") pod \"maas-api-e2e-derive-052492e6-7757568586-2k45z\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.137141 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.137006    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rbj2\" (UniqueName: \"kubernetes.io/projected/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-kube-api-access-5rbj2\") pod \"maas-api-e2e-derive-052492e6-7757568586-2k45z\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.137141 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.137067    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-openshift-service-ca\") pod \"maas-api-e2e-derive-052492e6-7757568586-2k45z\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.137749 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.137688    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-openshift-service-ca\") pod \"maas-api-e2e-derive-052492e6-7757568586-2k45z\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.139473 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.139454    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-maas-api-tls\") pod \"maas-api-e2e-derive-052492e6-7757568586-2k45z\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.148177 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.148152    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rbj2\" (UniqueName: \"kubernetes.io/projected/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-kube-api-access-5rbj2\") pod \"maas-api-e2e-derive-052492e6-7757568586-2k45z\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.245060 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.245004    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:41.380213 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.380179    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"]
Jul 30 20:59:41.381563 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:59:41.381539    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8859c61_0ca7_4eb9_ac7b_71cc324c685f.slice/crio-9227fe3076b77bc5d167869b2f178593c51a552498568f9d16a56fdf7f5b8909 WatchSource:0}: Error finding container 9227fe3076b77bc5d167869b2f178593c51a552498568f9d16a56fdf7f5b8909: Status 404 returned error can't find the container with id 9227fe3076b77bc5d167869b2f178593c51a552498568f9d16a56fdf7f5b8909
Jul 30 20:59:41.596372 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.596339    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"]
Jul 30 20:59:41.601663 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.601641    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:41.604944 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.604920    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-052492e6\""
Jul 30 20:59:41.605046 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.604942    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-052492e6-dockercfg-6br9z\""
Jul 30 20:59:41.607450 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.607425    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"]
Jul 30 20:59:41.618350 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.618329    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"]
Jul 30 20:59:41.622197 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.622181    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 20:59:41.637485 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.637464    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"]
Jul 30 20:59:41.679231 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.679202    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z" event={"ID":"f8859c61-0ca7-4eb9-ac7b-71cc324c685f","Type":"ContainerStarted","Data":"9227fe3076b77bc5d167869b2f178593c51a552498568f9d16a56fdf7f5b8909"}
Jul 30 20:59:41.744022 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.743990    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5v5w\" (UniqueName: \"kubernetes.io/projected/5b490b58-d869-4a9c-9589-3466cd401d79-kube-api-access-x5v5w\") pod \"payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq\" (UID: \"5b490b58-d869-4a9c-9589-3466cd401d79\") " pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 20:59:41.744146 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.744040    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8fc7c7-3966-47c8-a8b5-0348236aef91-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4\" (UID: \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:41.744146 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.744095    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b490b58-d869-4a9c-9589-3466cd401d79-plugins-config-volume\") pod \"payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq\" (UID: \"5b490b58-d869-4a9c-9589-3466cd401d79\") " pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 20:59:41.744224 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.744149    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl5l7\" (UniqueName: \"kubernetes.io/projected/0a8fc7c7-3966-47c8-a8b5-0348236aef91-kube-api-access-pl5l7\") pod \"payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4\" (UID: \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:41.845028 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.845000    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8fc7c7-3966-47c8-a8b5-0348236aef91-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4\" (UID: \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:41.845141 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.845038    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b490b58-d869-4a9c-9589-3466cd401d79-plugins-config-volume\") pod \"payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq\" (UID: \"5b490b58-d869-4a9c-9589-3466cd401d79\") " pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 20:59:41.845191 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.845172    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl5l7\" (UniqueName: \"kubernetes.io/projected/0a8fc7c7-3966-47c8-a8b5-0348236aef91-kube-api-access-pl5l7\") pod \"payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4\" (UID: \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:41.845261 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.845244    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5v5w\" (UniqueName: \"kubernetes.io/projected/5b490b58-d869-4a9c-9589-3466cd401d79-kube-api-access-x5v5w\") pod \"payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq\" (UID: \"5b490b58-d869-4a9c-9589-3466cd401d79\") " pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 20:59:41.845649 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.845630    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8fc7c7-3966-47c8-a8b5-0348236aef91-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4\" (UID: \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:41.845742 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.845652    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b490b58-d869-4a9c-9589-3466cd401d79-plugins-config-volume\") pod \"payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq\" (UID: \"5b490b58-d869-4a9c-9589-3466cd401d79\") " pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 20:59:41.854154 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.854100    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl5l7\" (UniqueName: \"kubernetes.io/projected/0a8fc7c7-3966-47c8-a8b5-0348236aef91-kube-api-access-pl5l7\") pod \"payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4\" (UID: \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:41.854444 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.854420    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5v5w\" (UniqueName: \"kubernetes.io/projected/5b490b58-d869-4a9c-9589-3466cd401d79-kube-api-access-x5v5w\") pod \"payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq\" (UID: \"5b490b58-d869-4a9c-9589-3466cd401d79\") " pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 20:59:41.914372 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.914355    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:41.932398 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:41.932378    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 20:59:42.064295 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.064265    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"]
Jul 30 20:59:42.065926 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:59:42.065894    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8fc7c7_3966_47c8_a8b5_0348236aef91.slice/crio-7165932e1496a4134d0573f5c62a08af93d672df8c6900bd0c4210101025482e WatchSource:0}: Error finding container 7165932e1496a4134d0573f5c62a08af93d672df8c6900bd0c4210101025482e: Status 404 returned error can't find the container with id 7165932e1496a4134d0573f5c62a08af93d672df8c6900bd0c4210101025482e
Jul 30 20:59:42.090423 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.090401    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"]
Jul 30 20:59:42.091687 ip-10-0-134-148 kubenswrapper[2547]: W0730 20:59:42.091663    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b490b58_d869_4a9c_9589_3466cd401d79.slice/crio-a4a012ca7fae534c469ac6f48e85570a46a748cb54fb37ea95afcc467e71d333 WatchSource:0}: Error finding container a4a012ca7fae534c469ac6f48e85570a46a748cb54fb37ea95afcc467e71d333: Status 404 returned error can't find the container with id a4a012ca7fae534c469ac6f48e85570a46a748cb54fb37ea95afcc467e71d333
Jul 30 20:59:42.685624 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.685550    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z" event={"ID":"f8859c61-0ca7-4eb9-ac7b-71cc324c685f","Type":"ContainerStarted","Data":"1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43"}
Jul 30 20:59:42.685812 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.685724    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:42.687112 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.687094    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq" event={"ID":"5b490b58-d869-4a9c-9589-3466cd401d79","Type":"ContainerStarted","Data":"2943e84f52695d00c64bada5d53bddcd23257ca5a3c5f44c7919a62ab901ef5a"}
Jul 30 20:59:42.687176 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.687135    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq" event={"ID":"5b490b58-d869-4a9c-9589-3466cd401d79","Type":"ContainerStarted","Data":"a4a012ca7fae534c469ac6f48e85570a46a748cb54fb37ea95afcc467e71d333"}
Jul 30 20:59:42.687176 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.687163    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 20:59:42.688639 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.688618    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4" event={"ID":"0a8fc7c7-3966-47c8-a8b5-0348236aef91","Type":"ContainerStarted","Data":"445487f335d187e6422c6681e26cad4c2a921888f2532ce1f1a5c1ca13e3282a"}
Jul 30 20:59:42.688759 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.688682    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4" event={"ID":"0a8fc7c7-3966-47c8-a8b5-0348236aef91","Type":"ContainerStarted","Data":"7165932e1496a4134d0573f5c62a08af93d672df8c6900bd0c4210101025482e"}
Jul 30 20:59:42.688834 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.688773    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:42.707422 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.707374    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z" podStartSLOduration=2.207605179 podStartE2EDuration="2.707342076s" totalImagesPullingTime="499.736897ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:42.704323419 +0000 UTC m=+2061.165282231" watchObservedRunningTime="2026-07-30 20:59:42.707342076 +0000 UTC m=+2061.168300889"
Jul 30 20:59:42.723158 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.723117    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq" podStartSLOduration=1.723105747 podStartE2EDuration="1.723105747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:42.721492238 +0000 UTC m=+2061.182451052" watchObservedRunningTime="2026-07-30 20:59:42.723105747 +0000 UTC m=+2061.184064557"
Jul 30 20:59:42.737483 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:42.737441    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4" podStartSLOduration=1.737433454 podStartE2EDuration="1.737433454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:59:42.737168647 +0000 UTC m=+2061.198127458" watchObservedRunningTime="2026-07-30 20:59:42.737433454 +0000 UTC m=+2061.198392263"
Jul 30 20:59:48.700582 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:48.700554    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 20:59:53.696243 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:53.696215    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 20:59:53.696666 ip-10-0-134-148 kubenswrapper[2547]: I0730 20:59:53.696281    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 21:00:11.462437 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.462405    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"]
Jul 30 21:00:11.462924 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.462648    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z" podUID="f8859c61-0ca7-4eb9-ac7b-71cc324c685f" containerName="maas-api" containerID="cri-o://1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43" gracePeriod=30
Jul 30 21:00:11.607422 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.607205    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"]
Jul 30 21:00:11.607585 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.607459    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4" podUID="0a8fc7c7-3966-47c8-a8b5-0348236aef91" containerName="payload-pre-processing" containerID="cri-o://445487f335d187e6422c6681e26cad4c2a921888f2532ce1f1a5c1ca13e3282a" gracePeriod=30
Jul 30 21:00:11.610343 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.610315    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"]
Jul 30 21:00:11.610578 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.610546    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq" podUID="5b490b58-d869-4a9c-9589-3466cd401d79" containerName="payload-processing" containerID="cri-o://2943e84f52695d00c64bada5d53bddcd23257ca5a3c5f44c7919a62ab901ef5a" gracePeriod=30
Jul 30 21:00:11.744954 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.744923    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 21:00:11.835637 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.835323    2547 generic.go:356] "Generic (PLEG): container finished" podID="f8859c61-0ca7-4eb9-ac7b-71cc324c685f" containerID="1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43" exitCode=0
Jul 30 21:00:11.835637 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.835365    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z" event={"ID":"f8859c61-0ca7-4eb9-ac7b-71cc324c685f","Type":"ContainerDied","Data":"1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43"}
Jul 30 21:00:11.835637 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.835442    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z" event={"ID":"f8859c61-0ca7-4eb9-ac7b-71cc324c685f","Type":"ContainerDied","Data":"9227fe3076b77bc5d167869b2f178593c51a552498568f9d16a56fdf7f5b8909"}
Jul 30 21:00:11.835637 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.835477    2547 scope.go:117] "RemoveContainer" containerID="1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43"
Jul 30 21:00:11.836257 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.836235    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"
Jul 30 21:00:11.841362 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.841337    2547 generic.go:356] "Generic (PLEG): container finished" podID="5b490b58-d869-4a9c-9589-3466cd401d79" containerID="2943e84f52695d00c64bada5d53bddcd23257ca5a3c5f44c7919a62ab901ef5a" exitCode=0
Jul 30 21:00:11.841483 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.841420    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq" event={"ID":"5b490b58-d869-4a9c-9589-3466cd401d79","Type":"ContainerDied","Data":"2943e84f52695d00c64bada5d53bddcd23257ca5a3c5f44c7919a62ab901ef5a"}
Jul 30 21:00:11.843506 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.843476    2547 generic.go:356] "Generic (PLEG): container finished" podID="0a8fc7c7-3966-47c8-a8b5-0348236aef91" containerID="445487f335d187e6422c6681e26cad4c2a921888f2532ce1f1a5c1ca13e3282a" exitCode=0
Jul 30 21:00:11.843628 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.843551    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4" event={"ID":"0a8fc7c7-3966-47c8-a8b5-0348236aef91","Type":"ContainerDied","Data":"445487f335d187e6422c6681e26cad4c2a921888f2532ce1f1a5c1ca13e3282a"}
Jul 30 21:00:11.851233 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.851211    2547 scope.go:117] "RemoveContainer" containerID="1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43"
Jul 30 21:00:11.851550 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:00:11.851525    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43\": container with ID starting with 1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43 not found: ID does not exist" containerID="1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43"
Jul 30 21:00:11.851673 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.851562    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43"} err="failed to get container status \"1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43\": rpc error: code = NotFound desc = could not find container \"1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43\": container with ID starting with 1592bc66ad01c57f26db3dc3df57940ffafb05825b0f630f4e1b4717591d3f43 not found: ID does not exist"
Jul 30 21:00:11.882120 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.882094    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-maas-api-tls\") pod \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") "
Jul 30 21:00:11.882199 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.882137    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-openshift-service-ca\") pod \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") "
Jul 30 21:00:11.882199 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.882172    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rbj2\" (UniqueName: \"kubernetes.io/projected/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-kube-api-access-5rbj2\") pod \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\" (UID: \"f8859c61-0ca7-4eb9-ac7b-71cc324c685f\") "
Jul 30 21:00:11.882622 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.882593    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f8859c61-0ca7-4eb9-ac7b-71cc324c685f" (UID: "f8859c61-0ca7-4eb9-ac7b-71cc324c685f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:00:11.884292 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.884267    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-kube-api-access-5rbj2" (OuterVolumeSpecName: "kube-api-access-5rbj2") pod "f8859c61-0ca7-4eb9-ac7b-71cc324c685f" (UID: "f8859c61-0ca7-4eb9-ac7b-71cc324c685f"). InnerVolumeSpecName "kube-api-access-5rbj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:00:11.884292 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.884270    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f8859c61-0ca7-4eb9-ac7b-71cc324c685f" (UID: "f8859c61-0ca7-4eb9-ac7b-71cc324c685f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:00:11.903401 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.903382    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 21:00:11.905302 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.905284    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 21:00:11.984040 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.983969    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:00:11.984040 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.983993    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:00:11.984040 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:11.984003    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rbj2\" (UniqueName: \"kubernetes.io/projected/f8859c61-0ca7-4eb9-ac7b-71cc324c685f-kube-api-access-5rbj2\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:00:12.085047 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.085023    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl5l7\" (UniqueName: \"kubernetes.io/projected/0a8fc7c7-3966-47c8-a8b5-0348236aef91-kube-api-access-pl5l7\") pod \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\" (UID: \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\") "
Jul 30 21:00:12.085160 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.085073    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8fc7c7-3966-47c8-a8b5-0348236aef91-plugins-config-volume\") pod \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\" (UID: \"0a8fc7c7-3966-47c8-a8b5-0348236aef91\") "
Jul 30 21:00:12.085160 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.085126    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b490b58-d869-4a9c-9589-3466cd401d79-plugins-config-volume\") pod \"5b490b58-d869-4a9c-9589-3466cd401d79\" (UID: \"5b490b58-d869-4a9c-9589-3466cd401d79\") "
Jul 30 21:00:12.085257 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.085204    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5v5w\" (UniqueName: \"kubernetes.io/projected/5b490b58-d869-4a9c-9589-3466cd401d79-kube-api-access-x5v5w\") pod \"5b490b58-d869-4a9c-9589-3466cd401d79\" (UID: \"5b490b58-d869-4a9c-9589-3466cd401d79\") "
Jul 30 21:00:12.085427 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.085402    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a8fc7c7-3966-47c8-a8b5-0348236aef91-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a8fc7c7-3966-47c8-a8b5-0348236aef91" (UID: "0a8fc7c7-3966-47c8-a8b5-0348236aef91"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:00:12.085564 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.085537    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b490b58-d869-4a9c-9589-3466cd401d79-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b490b58-d869-4a9c-9589-3466cd401d79" (UID: "5b490b58-d869-4a9c-9589-3466cd401d79"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:00:12.087276 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.087257    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b490b58-d869-4a9c-9589-3466cd401d79-kube-api-access-x5v5w" (OuterVolumeSpecName: "kube-api-access-x5v5w") pod "5b490b58-d869-4a9c-9589-3466cd401d79" (UID: "5b490b58-d869-4a9c-9589-3466cd401d79"). InnerVolumeSpecName "kube-api-access-x5v5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:00:12.087276 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.087261    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a8fc7c7-3966-47c8-a8b5-0348236aef91-kube-api-access-pl5l7" (OuterVolumeSpecName: "kube-api-access-pl5l7") pod "0a8fc7c7-3966-47c8-a8b5-0348236aef91" (UID: "0a8fc7c7-3966-47c8-a8b5-0348236aef91"). InnerVolumeSpecName "kube-api-access-pl5l7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:00:12.154358 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.154333    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"]
Jul 30 21:00:12.163787 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.163755    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-052492e6-7757568586-2k45z"]
Jul 30 21:00:12.186570 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.186553    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5v5w\" (UniqueName: \"kubernetes.io/projected/5b490b58-d869-4a9c-9589-3466cd401d79-kube-api-access-x5v5w\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:00:12.186570 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.186571    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl5l7\" (UniqueName: \"kubernetes.io/projected/0a8fc7c7-3966-47c8-a8b5-0348236aef91-kube-api-access-pl5l7\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:00:12.186740 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.186581    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a8fc7c7-3966-47c8-a8b5-0348236aef91-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:00:12.186740 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.186595    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b490b58-d869-4a9c-9589-3466cd401d79-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:00:12.850893 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.850862    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"
Jul 30 21:00:12.851325 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.850863    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq" event={"ID":"5b490b58-d869-4a9c-9589-3466cd401d79","Type":"ContainerDied","Data":"a4a012ca7fae534c469ac6f48e85570a46a748cb54fb37ea95afcc467e71d333"}
Jul 30 21:00:12.851325 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.850992    2547 scope.go:117] "RemoveContainer" containerID="2943e84f52695d00c64bada5d53bddcd23257ca5a3c5f44c7919a62ab901ef5a"
Jul 30 21:00:12.852349 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.852316    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4" event={"ID":"0a8fc7c7-3966-47c8-a8b5-0348236aef91","Type":"ContainerDied","Data":"7165932e1496a4134d0573f5c62a08af93d672df8c6900bd0c4210101025482e"}
Jul 30 21:00:12.852349 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.852322    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"
Jul 30 21:00:12.859828 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.859806    2547 scope.go:117] "RemoveContainer" containerID="445487f335d187e6422c6681e26cad4c2a921888f2532ce1f1a5c1ca13e3282a"
Jul 30 21:00:12.882084 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.882060    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"]
Jul 30 21:00:12.883905 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.883885    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-052492e6-648d5c8f58-8vkwq"]
Jul 30 21:00:12.902387 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.902361    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"]
Jul 30 21:00:12.904468 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:12.904450    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-052492e6-5f89796944-s7vw4"]
Jul 30 21:00:14.073244 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:14.073210    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a8fc7c7-3966-47c8-a8b5-0348236aef91" path="/var/lib/kubelet/pods/0a8fc7c7-3966-47c8-a8b5-0348236aef91/volumes"
Jul 30 21:00:14.073600 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:14.073540    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b490b58-d869-4a9c-9589-3466cd401d79" path="/var/lib/kubelet/pods/5b490b58-d869-4a9c-9589-3466cd401d79/volumes"
Jul 30 21:00:14.073907 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:14.073894    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8859c61-0ca7-4eb9-ac7b-71cc324c685f" path="/var/lib/kubelet/pods/f8859c61-0ca7-4eb9-ac7b-71cc324c685f/volumes"
Jul 30 21:00:22.162002 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:22.161972    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:00:22.165934 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:22.165911    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:00:22.166883 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:22.166862    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:00:22.170344 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:00:22.170328    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:02:15.290664 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.290636    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"]
Jul 30 21:02:15.291254 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.291223    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b490b58-d869-4a9c-9589-3466cd401d79" containerName="payload-processing"
Jul 30 21:02:15.291254 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.291245    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b490b58-d869-4a9c-9589-3466cd401d79" containerName="payload-processing"
Jul 30 21:02:15.291380 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.291267    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8859c61-0ca7-4eb9-ac7b-71cc324c685f" containerName="maas-api"
Jul 30 21:02:15.291380 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.291275    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8859c61-0ca7-4eb9-ac7b-71cc324c685f" containerName="maas-api"
Jul 30 21:02:15.291380 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.291305    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a8fc7c7-3966-47c8-a8b5-0348236aef91" containerName="payload-pre-processing"
Jul 30 21:02:15.291380 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.291313    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a8fc7c7-3966-47c8-a8b5-0348236aef91" containerName="payload-pre-processing"
Jul 30 21:02:15.291571 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.291412    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a8fc7c7-3966-47c8-a8b5-0348236aef91" containerName="payload-pre-processing"
Jul 30 21:02:15.291571 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.291425    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b490b58-d869-4a9c-9589-3466cd401d79" containerName="payload-processing"
Jul 30 21:02:15.291571 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.291435    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8859c61-0ca7-4eb9-ac7b-71cc324c685f" containerName="maas-api"
Jul 30 21:02:15.294831 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.294809    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.297304 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.297284    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-577a5369-serving-cert\""
Jul 30 21:02:15.305690 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.305669    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"]
Jul 30 21:02:15.473985 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.473956    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7124104-d1dd-462f-a8af-d7839e10a54d-openshift-service-ca\") pod \"maas-api-e2e-mt-577a5369-7b96dd476-cktcp\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.474156 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.474070    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8cvl\" (UniqueName: \"kubernetes.io/projected/f7124104-d1dd-462f-a8af-d7839e10a54d-kube-api-access-g8cvl\") pod \"maas-api-e2e-mt-577a5369-7b96dd476-cktcp\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.474156 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.474144    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7124104-d1dd-462f-a8af-d7839e10a54d-maas-api-tls\") pod \"maas-api-e2e-mt-577a5369-7b96dd476-cktcp\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.574746 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.574723    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8cvl\" (UniqueName: \"kubernetes.io/projected/f7124104-d1dd-462f-a8af-d7839e10a54d-kube-api-access-g8cvl\") pod \"maas-api-e2e-mt-577a5369-7b96dd476-cktcp\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.574891 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.574780    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7124104-d1dd-462f-a8af-d7839e10a54d-maas-api-tls\") pod \"maas-api-e2e-mt-577a5369-7b96dd476-cktcp\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.574891 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.574826    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7124104-d1dd-462f-a8af-d7839e10a54d-openshift-service-ca\") pod \"maas-api-e2e-mt-577a5369-7b96dd476-cktcp\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.575496 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.575478    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7124104-d1dd-462f-a8af-d7839e10a54d-openshift-service-ca\") pod \"maas-api-e2e-mt-577a5369-7b96dd476-cktcp\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.577374 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.577353    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7124104-d1dd-462f-a8af-d7839e10a54d-maas-api-tls\") pod \"maas-api-e2e-mt-577a5369-7b96dd476-cktcp\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.585413 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.585393    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8cvl\" (UniqueName: \"kubernetes.io/projected/f7124104-d1dd-462f-a8af-d7839e10a54d-kube-api-access-g8cvl\") pod \"maas-api-e2e-mt-577a5369-7b96dd476-cktcp\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.608237 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.608211    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:15.745063 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.745041    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"]
Jul 30 21:02:15.747075 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:02:15.747053    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7124104_d1dd_462f_a8af_d7839e10a54d.slice/crio-d34187826c00420774d414df4ec82e63ec2edd019d6638cb1508f7fa9f9b82de WatchSource:0}: Error finding container d34187826c00420774d414df4ec82e63ec2edd019d6638cb1508f7fa9f9b82de: Status 404 returned error can't find the container with id d34187826c00420774d414df4ec82e63ec2edd019d6638cb1508f7fa9f9b82de
Jul 30 21:02:15.953529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.953449    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"]
Jul 30 21:02:15.964535 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.964506    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:15.969985 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.969958    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"]
Jul 30 21:02:15.970642 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.970616    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-577a5369-dockercfg-95ph7\""
Jul 30 21:02:15.970802 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.970740    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-577a5369\""
Jul 30 21:02:15.977189 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.977166    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"]
Jul 30 21:02:15.980676 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.980658    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:15.993519 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:15.993495    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"]
Jul 30 21:02:16.079144 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.079112    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2bg4\" (UniqueName: \"kubernetes.io/projected/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-kube-api-access-b2bg4\") pod \"payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj\" (UID: \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:16.079144 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.079143    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-plugins-config-volume\") pod \"payload-processing-e2e-mt-577a5369-5799867db-kf2sg\" (UID: \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\") " pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:16.079322 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.079169    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqkbv\" (UniqueName: \"kubernetes.io/projected/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-kube-api-access-hqkbv\") pod \"payload-processing-e2e-mt-577a5369-5799867db-kf2sg\" (UID: \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\") " pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:16.079322 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.079296    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj\" (UID: \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:16.179989 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.179956    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj\" (UID: \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:16.180347 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.180324    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bg4\" (UniqueName: \"kubernetes.io/projected/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-kube-api-access-b2bg4\") pod \"payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj\" (UID: \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:16.180467 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.180365    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-plugins-config-volume\") pod \"payload-processing-e2e-mt-577a5369-5799867db-kf2sg\" (UID: \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\") " pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:16.180467 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.180429    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqkbv\" (UniqueName: \"kubernetes.io/projected/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-kube-api-access-hqkbv\") pod \"payload-processing-e2e-mt-577a5369-5799867db-kf2sg\" (UID: \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\") " pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:16.180840 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.180820    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj\" (UID: \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:16.181088 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.181071    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-plugins-config-volume\") pod \"payload-processing-e2e-mt-577a5369-5799867db-kf2sg\" (UID: \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\") " pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:16.190012 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.189962    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2bg4\" (UniqueName: \"kubernetes.io/projected/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-kube-api-access-b2bg4\") pod \"payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj\" (UID: \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:16.190904 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.190885    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqkbv\" (UniqueName: \"kubernetes.io/projected/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-kube-api-access-hqkbv\") pod \"payload-processing-e2e-mt-577a5369-5799867db-kf2sg\" (UID: \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\") " pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:16.278406 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.278352    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:16.290909 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.290890    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:16.437427 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.437272    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"]
Jul 30 21:02:16.440873 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:02:16.440835    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod543aed78_2dfb_4b93_9d28_8e7b2b4f4904.slice/crio-50fa186587813f35cc57ef05e2625cd83e3bde738273be1e4569a507f7afaca5 WatchSource:0}: Error finding container 50fa186587813f35cc57ef05e2625cd83e3bde738273be1e4569a507f7afaca5: Status 404 returned error can't find the container with id 50fa186587813f35cc57ef05e2625cd83e3bde738273be1e4569a507f7afaca5
Jul 30 21:02:16.461151 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.461120    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj" event={"ID":"543aed78-2dfb-4b93-9d28-8e7b2b4f4904","Type":"ContainerStarted","Data":"50fa186587813f35cc57ef05e2625cd83e3bde738273be1e4569a507f7afaca5"}
Jul 30 21:02:16.462714 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.462670    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp" event={"ID":"f7124104-d1dd-462f-a8af-d7839e10a54d","Type":"ContainerStarted","Data":"35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97"}
Jul 30 21:02:16.462820 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.462739    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp" event={"ID":"f7124104-d1dd-462f-a8af-d7839e10a54d","Type":"ContainerStarted","Data":"d34187826c00420774d414df4ec82e63ec2edd019d6638cb1508f7fa9f9b82de"}
Jul 30 21:02:16.462939 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.462849    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:16.466398 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.466366    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"]
Jul 30 21:02:16.469145 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:02:16.469126    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c07f1ce_c36e_4b23_8879_9bbbe9efa8ec.slice/crio-cae6ab791ac02ff5da9e6b38f2596a6c689d3dfef7062e7c4d09b40a6db4cd87 WatchSource:0}: Error finding container cae6ab791ac02ff5da9e6b38f2596a6c689d3dfef7062e7c4d09b40a6db4cd87: Status 404 returned error can't find the container with id cae6ab791ac02ff5da9e6b38f2596a6c689d3dfef7062e7c4d09b40a6db4cd87
Jul 30 21:02:16.480005 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:16.479967    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp" podStartSLOduration=0.9421494 podStartE2EDuration="1.47995292s" totalImagesPullingTime="537.80352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:02:16.478888658 +0000 UTC m=+2214.939847470" watchObservedRunningTime="2026-07-30 21:02:16.47995292 +0000 UTC m=+2214.940911730"
Jul 30 21:02:17.471630 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:17.471596    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg" event={"ID":"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec","Type":"ContainerStarted","Data":"f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87"}
Jul 30 21:02:17.472061 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:17.471655    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:17.472061 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:17.471672    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg" event={"ID":"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec","Type":"ContainerStarted","Data":"cae6ab791ac02ff5da9e6b38f2596a6c689d3dfef7062e7c4d09b40a6db4cd87"}
Jul 30 21:02:17.473099 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:17.473073    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj" event={"ID":"543aed78-2dfb-4b93-9d28-8e7b2b4f4904","Type":"ContainerStarted","Data":"c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054"}
Jul 30 21:02:17.473296 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:17.473281    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:17.489020 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:17.488979    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg" podStartSLOduration=2.488965081 podStartE2EDuration="2.488965081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:02:17.487798165 +0000 UTC m=+2215.948756978" watchObservedRunningTime="2026-07-30 21:02:17.488965081 +0000 UTC m=+2215.949923891"
Jul 30 21:02:17.503340 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:17.503299    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj" podStartSLOduration=2.503287035 podStartE2EDuration="2.503287035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:02:17.501569834 +0000 UTC m=+2215.962528646" watchObservedRunningTime="2026-07-30 21:02:17.503287035 +0000 UTC m=+2215.964245846"
Jul 30 21:02:22.478100 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:22.478070    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:28.480337 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:28.480308    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:28.480730 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:28.480374    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:56.651091 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:56.651009    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"]
Jul 30 21:02:56.651645 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:56.651249    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp" podUID="f7124104-d1dd-462f-a8af-d7839e10a54d" containerName="maas-api" containerID="cri-o://35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97" gracePeriod=30
Jul 30 21:02:56.702998 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:56.702966    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"]
Jul 30 21:02:56.703242 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:56.703192    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg" podUID="8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec" containerName="payload-processing" containerID="cri-o://f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87" gracePeriod=30
Jul 30 21:02:56.712775 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:56.712747    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"]
Jul 30 21:02:56.712992 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:56.712964    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj" podUID="543aed78-2dfb-4b93-9d28-8e7b2b4f4904" containerName="payload-pre-processing" containerID="cri-o://c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054" gracePeriod=30
Jul 30 21:02:56.974114 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:56.974085    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:56.984856 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:56.984834    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:56.995090 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:56.995038    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:57.091260 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091238    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2bg4\" (UniqueName: \"kubernetes.io/projected/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-kube-api-access-b2bg4\") pod \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\" (UID: \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\") "
Jul 30 21:02:57.091407 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091267    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-plugins-config-volume\") pod \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\" (UID: \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\") "
Jul 30 21:02:57.091407 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091309    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7124104-d1dd-462f-a8af-d7839e10a54d-openshift-service-ca\") pod \"f7124104-d1dd-462f-a8af-d7839e10a54d\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") "
Jul 30 21:02:57.091407 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091328    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8cvl\" (UniqueName: \"kubernetes.io/projected/f7124104-d1dd-462f-a8af-d7839e10a54d-kube-api-access-g8cvl\") pod \"f7124104-d1dd-462f-a8af-d7839e10a54d\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") "
Jul 30 21:02:57.091534 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091448    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqkbv\" (UniqueName: \"kubernetes.io/projected/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-kube-api-access-hqkbv\") pod \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\" (UID: \"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec\") "
Jul 30 21:02:57.091584 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091568    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7124104-d1dd-462f-a8af-d7839e10a54d-maas-api-tls\") pod \"f7124104-d1dd-462f-a8af-d7839e10a54d\" (UID: \"f7124104-d1dd-462f-a8af-d7839e10a54d\") "
Jul 30 21:02:57.091642 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091614    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-plugins-config-volume\") pod \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\" (UID: \"543aed78-2dfb-4b93-9d28-8e7b2b4f4904\") "
Jul 30 21:02:57.091715 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091652    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7124104-d1dd-462f-a8af-d7839e10a54d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f7124104-d1dd-462f-a8af-d7839e10a54d" (UID: "f7124104-d1dd-462f-a8af-d7839e10a54d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:02:57.091715 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091663    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec" (UID: "8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:02:57.091977 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091917    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:02:57.091977 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.091941    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7124104-d1dd-462f-a8af-d7839e10a54d-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:02:57.092194 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.092165    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "543aed78-2dfb-4b93-9d28-8e7b2b4f4904" (UID: "543aed78-2dfb-4b93-9d28-8e7b2b4f4904"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:02:57.093536 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.093511    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-kube-api-access-b2bg4" (OuterVolumeSpecName: "kube-api-access-b2bg4") pod "543aed78-2dfb-4b93-9d28-8e7b2b4f4904" (UID: "543aed78-2dfb-4b93-9d28-8e7b2b4f4904"). InnerVolumeSpecName "kube-api-access-b2bg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:02:57.093638 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.093537    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-kube-api-access-hqkbv" (OuterVolumeSpecName: "kube-api-access-hqkbv") pod "8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec" (UID: "8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec"). InnerVolumeSpecName "kube-api-access-hqkbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:02:57.093731 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.093674    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7124104-d1dd-462f-a8af-d7839e10a54d-kube-api-access-g8cvl" (OuterVolumeSpecName: "kube-api-access-g8cvl") pod "f7124104-d1dd-462f-a8af-d7839e10a54d" (UID: "f7124104-d1dd-462f-a8af-d7839e10a54d"). InnerVolumeSpecName "kube-api-access-g8cvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:02:57.093902 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.093883    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7124104-d1dd-462f-a8af-d7839e10a54d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f7124104-d1dd-462f-a8af-d7839e10a54d" (UID: "f7124104-d1dd-462f-a8af-d7839e10a54d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:02:57.193046 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.192997    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8cvl\" (UniqueName: \"kubernetes.io/projected/f7124104-d1dd-462f-a8af-d7839e10a54d-kube-api-access-g8cvl\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:02:57.193046 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.193018    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqkbv\" (UniqueName: \"kubernetes.io/projected/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec-kube-api-access-hqkbv\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:02:57.193046 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.193030    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7124104-d1dd-462f-a8af-d7839e10a54d-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:02:57.193046 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.193039    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:02:57.193236 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.193049    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2bg4\" (UniqueName: \"kubernetes.io/projected/543aed78-2dfb-4b93-9d28-8e7b2b4f4904-kube-api-access-b2bg4\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:02:57.676046 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.676013    2547 generic.go:356] "Generic (PLEG): container finished" podID="8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec" containerID="f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87" exitCode=0
Jul 30 21:02:57.676388 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.676054    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"
Jul 30 21:02:57.676388 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.676103    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg" event={"ID":"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec","Type":"ContainerDied","Data":"f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87"}
Jul 30 21:02:57.676388 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.676151    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg" event={"ID":"8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec","Type":"ContainerDied","Data":"cae6ab791ac02ff5da9e6b38f2596a6c689d3dfef7062e7c4d09b40a6db4cd87"}
Jul 30 21:02:57.676388 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.676173    2547 scope.go:117] "RemoveContainer" containerID="f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87"
Jul 30 21:02:57.677412 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.677367    2547 generic.go:356] "Generic (PLEG): container finished" podID="543aed78-2dfb-4b93-9d28-8e7b2b4f4904" containerID="c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054" exitCode=0
Jul 30 21:02:57.677480 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.677415    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"
Jul 30 21:02:57.677480 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.677451    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj" event={"ID":"543aed78-2dfb-4b93-9d28-8e7b2b4f4904","Type":"ContainerDied","Data":"c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054"}
Jul 30 21:02:57.677588 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.677514    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj" event={"ID":"543aed78-2dfb-4b93-9d28-8e7b2b4f4904","Type":"ContainerDied","Data":"50fa186587813f35cc57ef05e2625cd83e3bde738273be1e4569a507f7afaca5"}
Jul 30 21:02:57.678889 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.678866    2547 generic.go:356] "Generic (PLEG): container finished" podID="f7124104-d1dd-462f-a8af-d7839e10a54d" containerID="35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97" exitCode=0
Jul 30 21:02:57.679144 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.678915    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"
Jul 30 21:02:57.679144 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.678940    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp" event={"ID":"f7124104-d1dd-462f-a8af-d7839e10a54d","Type":"ContainerDied","Data":"35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97"}
Jul 30 21:02:57.679144 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.678990    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp" event={"ID":"f7124104-d1dd-462f-a8af-d7839e10a54d","Type":"ContainerDied","Data":"d34187826c00420774d414df4ec82e63ec2edd019d6638cb1508f7fa9f9b82de"}
Jul 30 21:02:57.686254 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.686234    2547 scope.go:117] "RemoveContainer" containerID="f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87"
Jul 30 21:02:57.686522 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:02:57.686502    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87\": container with ID starting with f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87 not found: ID does not exist" containerID="f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87"
Jul 30 21:02:57.686615 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.686527    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87"} err="failed to get container status \"f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87\": rpc error: code = NotFound desc = could not find container \"f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87\": container with ID starting with f48ce69b7461b88c880f907d317f86f6e15d6b47c1b4725d0414d87255bd2e87 not found: ID does not exist"
Jul 30 21:02:57.686615 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.686544    2547 scope.go:117] "RemoveContainer" containerID="c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054"
Jul 30 21:02:57.694982 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.694961    2547 scope.go:117] "RemoveContainer" containerID="c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054"
Jul 30 21:02:57.695236 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:02:57.695219    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054\": container with ID starting with c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054 not found: ID does not exist" containerID="c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054"
Jul 30 21:02:57.695289 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.695243    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054"} err="failed to get container status \"c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054\": rpc error: code = NotFound desc = could not find container \"c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054\": container with ID starting with c000fca0475fd526b26a2c52d3fa3ac6ea28feb6b0e9b321ac690e975708d054 not found: ID does not exist"
Jul 30 21:02:57.695289 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.695259    2547 scope.go:117] "RemoveContainer" containerID="35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97"
Jul 30 21:02:57.702361 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.702330    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"]
Jul 30 21:02:57.703156 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.703139    2547 scope.go:117] "RemoveContainer" containerID="35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97"
Jul 30 21:02:57.703396 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:02:57.703379    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97\": container with ID starting with 35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97 not found: ID does not exist" containerID="35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97"
Jul 30 21:02:57.703466 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.703403    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97"} err="failed to get container status \"35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97\": rpc error: code = NotFound desc = could not find container \"35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97\": container with ID starting with 35799981d7035cd0cba2e6a440f15a8d93929d0982e7a42925bf1d01d0956c97 not found: ID does not exist"
Jul 30 21:02:57.705293 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.705272    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-577a5369-5799867db-kf2sg"]
Jul 30 21:02:57.716885 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.716865    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"]
Jul 30 21:02:57.719865 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.719845    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-577a5369-7667d5db9b-gvjpj"]
Jul 30 21:02:57.736272 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.731191    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"]
Jul 30 21:02:57.736272 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:57.736052    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-577a5369-7b96dd476-cktcp"]
Jul 30 21:02:58.073729 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:58.073684    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="543aed78-2dfb-4b93-9d28-8e7b2b4f4904" path="/var/lib/kubelet/pods/543aed78-2dfb-4b93-9d28-8e7b2b4f4904/volumes"
Jul 30 21:02:58.074038 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:58.074026    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec" path="/var/lib/kubelet/pods/8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec/volumes"
Jul 30 21:02:58.074350 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:02:58.074339    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7124104-d1dd-462f-a8af-d7839e10a54d" path="/var/lib/kubelet/pods/f7124104-d1dd-462f-a8af-d7839e10a54d/volumes"
Jul 30 21:03:44.237689 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.237657    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"]
Jul 30 21:03:44.238208 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.237853    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd" podUID="c9268292-7944-4e0a-81fa-1d955c58878c" containerName="manager" containerID="cri-o://9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b" gracePeriod=10
Jul 30 21:03:44.496376 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.496317    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 21:03:44.656236 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.656204    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9268292-7944-4e0a-81fa-1d955c58878c-extensions-socket-volume\") pod \"c9268292-7944-4e0a-81fa-1d955c58878c\" (UID: \"c9268292-7944-4e0a-81fa-1d955c58878c\") "
Jul 30 21:03:44.656376 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.656299    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph97l\" (UniqueName: \"kubernetes.io/projected/c9268292-7944-4e0a-81fa-1d955c58878c-kube-api-access-ph97l\") pod \"c9268292-7944-4e0a-81fa-1d955c58878c\" (UID: \"c9268292-7944-4e0a-81fa-1d955c58878c\") "
Jul 30 21:03:44.656599 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.656571    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9268292-7944-4e0a-81fa-1d955c58878c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c9268292-7944-4e0a-81fa-1d955c58878c" (UID: "c9268292-7944-4e0a-81fa-1d955c58878c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 21:03:44.658372 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.658352    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9268292-7944-4e0a-81fa-1d955c58878c-kube-api-access-ph97l" (OuterVolumeSpecName: "kube-api-access-ph97l") pod "c9268292-7944-4e0a-81fa-1d955c58878c" (UID: "c9268292-7944-4e0a-81fa-1d955c58878c"). InnerVolumeSpecName "kube-api-access-ph97l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:03:44.756987 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.756932    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph97l\" (UniqueName: \"kubernetes.io/projected/c9268292-7944-4e0a-81fa-1d955c58878c-kube-api-access-ph97l\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:03:44.756987 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.756952    2547 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9268292-7944-4e0a-81fa-1d955c58878c-extensions-socket-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:03:44.903733 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.903684    2547 generic.go:356] "Generic (PLEG): container finished" podID="c9268292-7944-4e0a-81fa-1d955c58878c" containerID="9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b" exitCode=0
Jul 30 21:03:44.903896 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.903748    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"
Jul 30 21:03:44.903896 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.903779    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd" event={"ID":"c9268292-7944-4e0a-81fa-1d955c58878c","Type":"ContainerDied","Data":"9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b"}
Jul 30 21:03:44.903896 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.903855    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd" event={"ID":"c9268292-7944-4e0a-81fa-1d955c58878c","Type":"ContainerDied","Data":"569987b2648e761ec9ba5d63ad58098b87abb8f330a65258fade3f13859ce710"}
Jul 30 21:03:44.903896 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.903880    2547 scope.go:117] "RemoveContainer" containerID="9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b"
Jul 30 21:03:44.912913 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.912893    2547 scope.go:117] "RemoveContainer" containerID="9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b"
Jul 30 21:03:44.913171 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:03:44.913155    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b\": container with ID starting with 9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b not found: ID does not exist" containerID="9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b"
Jul 30 21:03:44.913220 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.913179    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b"} err="failed to get container status \"9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b\": rpc error: code = NotFound desc = could not find container \"9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b\": container with ID starting with 9ffc20e56bc55ebcf72a07f24cd5f8b1fb406f5e99801a2f74c09e8c79b2e35b not found: ID does not exist"
Jul 30 21:03:44.925104 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.925083    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"]
Jul 30 21:03:44.929443 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:44.929422    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jhnfd"]
Jul 30 21:03:46.072491 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:03:46.072460    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9268292-7944-4e0a-81fa-1d955c58878c" path="/var/lib/kubelet/pods/c9268292-7944-4e0a-81fa-1d955c58878c/volumes"
Jul 30 21:04:59.879332 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879296    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"]
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879667    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec" containerName="payload-processing"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879678    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec" containerName="payload-processing"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879687    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7124104-d1dd-462f-a8af-d7839e10a54d" containerName="maas-api"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879693    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7124104-d1dd-462f-a8af-d7839e10a54d" containerName="maas-api"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879717    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9268292-7944-4e0a-81fa-1d955c58878c" containerName="manager"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879722    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9268292-7944-4e0a-81fa-1d955c58878c" containerName="manager"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879727    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="543aed78-2dfb-4b93-9d28-8e7b2b4f4904" containerName="payload-pre-processing"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879732    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="543aed78-2dfb-4b93-9d28-8e7b2b4f4904" containerName="payload-pre-processing"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879793    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7124104-d1dd-462f-a8af-d7839e10a54d" containerName="maas-api"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879801    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9268292-7944-4e0a-81fa-1d955c58878c" containerName="manager"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879809    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="543aed78-2dfb-4b93-9d28-8e7b2b4f4904" containerName="payload-pre-processing"
Jul 30 21:04:59.879855 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.879815    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c07f1ce-c36e-4b23-8879-9bbbe9efa8ec" containerName="payload-processing"
Jul 30 21:04:59.882563 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.882547    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:04:59.885293 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.885268    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4j9mr\""
Jul 30 21:04:59.894289 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:04:59.894270    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"]
Jul 30 21:05:00.019352 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:00.019326    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bce408fb-a78e-4138-b776-855e915eadb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-lv67d\" (UID: \"bce408fb-a78e-4138-b776-855e915eadb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:05:00.019446 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:00.019362    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd6bd\" (UniqueName: \"kubernetes.io/projected/bce408fb-a78e-4138-b776-855e915eadb8-kube-api-access-nd6bd\") pod \"kuadrant-operator-controller-manager-68cf589589-lv67d\" (UID: \"bce408fb-a78e-4138-b776-855e915eadb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:05:00.120374 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:00.120348    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bce408fb-a78e-4138-b776-855e915eadb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-lv67d\" (UID: \"bce408fb-a78e-4138-b776-855e915eadb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:05:00.120467 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:00.120385    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd6bd\" (UniqueName: \"kubernetes.io/projected/bce408fb-a78e-4138-b776-855e915eadb8-kube-api-access-nd6bd\") pod \"kuadrant-operator-controller-manager-68cf589589-lv67d\" (UID: \"bce408fb-a78e-4138-b776-855e915eadb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:05:00.120771 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:00.120754    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bce408fb-a78e-4138-b776-855e915eadb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-lv67d\" (UID: \"bce408fb-a78e-4138-b776-855e915eadb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:05:00.129778 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:00.129728    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd6bd\" (UniqueName: \"kubernetes.io/projected/bce408fb-a78e-4138-b776-855e915eadb8-kube-api-access-nd6bd\") pod \"kuadrant-operator-controller-manager-68cf589589-lv67d\" (UID: \"bce408fb-a78e-4138-b776-855e915eadb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:05:00.192046 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:00.192023    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:05:00.337179 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:00.337147    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"]
Jul 30 21:05:00.337428 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:05:00.337405    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce408fb_a78e_4138_b776_855e915eadb8.slice/crio-3c9561526c4fd21e1c090c270275c246ed83113063de7a37ed83ddfccccd6942 WatchSource:0}: Error finding container 3c9561526c4fd21e1c090c270275c246ed83113063de7a37ed83ddfccccd6942: Status 404 returned error can't find the container with id 3c9561526c4fd21e1c090c270275c246ed83113063de7a37ed83ddfccccd6942
Jul 30 21:05:01.251278 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:01.251238    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d" event={"ID":"bce408fb-a78e-4138-b776-855e915eadb8","Type":"ContainerStarted","Data":"1067c9d6f987acd1e5b419128072637ef943007d299c81691a70e6793004ab75"}
Jul 30 21:05:01.251651 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:01.251301    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d" event={"ID":"bce408fb-a78e-4138-b776-855e915eadb8","Type":"ContainerStarted","Data":"3c9561526c4fd21e1c090c270275c246ed83113063de7a37ed83ddfccccd6942"}
Jul 30 21:05:01.251651 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:01.251337    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:05:01.272497 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:01.272452    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d" podStartSLOduration=2.272437697 podStartE2EDuration="2.272437697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:04:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:05:01.269817503 +0000 UTC m=+2379.730776336" watchObservedRunningTime="2026-07-30 21:05:01.272437697 +0000 UTC m=+2379.733396509"
Jul 30 21:05:12.257649 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:12.257622    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lv67d"
Jul 30 21:05:22.192160 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:22.192117    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:05:22.196551 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:22.196528    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:05:22.201121 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:22.201100    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:05:22.205000 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:22.204984    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:05:23.053757 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.053726    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"]
Jul 30 21:05:23.058479 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.058457    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.060987 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.060968    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9e2107d7-serving-cert\""
Jul 30 21:05:23.067467 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.067416    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"]
Jul 30 21:05:23.102141 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.102113    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5xdc\" (UniqueName: \"kubernetes.io/projected/b6aaaf38-664b-4581-b2dc-879ffc983d6c-kube-api-access-s5xdc\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.102289 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.102181    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6aaaf38-664b-4581-b2dc-879ffc983d6c-maas-api-tls\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.102289 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.102230    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6aaaf38-664b-4581-b2dc-879ffc983d6c-openshift-service-ca\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.202740 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.202684    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5xdc\" (UniqueName: \"kubernetes.io/projected/b6aaaf38-664b-4581-b2dc-879ffc983d6c-kube-api-access-s5xdc\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.203111 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.202773    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6aaaf38-664b-4581-b2dc-879ffc983d6c-maas-api-tls\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.203111 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.202800    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6aaaf38-664b-4581-b2dc-879ffc983d6c-openshift-service-ca\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.203111 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:05:23.202912    2547 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-serving-cert: secret "maas-api-e2e-mt-9e2107d7-serving-cert" not found
Jul 30 21:05:23.203111 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:05:23.202977    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6aaaf38-664b-4581-b2dc-879ffc983d6c-maas-api-tls podName:b6aaaf38-664b-4581-b2dc-879ffc983d6c nodeName:}" failed. No retries permitted until 2026-07-30 21:05:23.702959901 +0000 UTC m=+2402.163918704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6aaaf38-664b-4581-b2dc-879ffc983d6c-maas-api-tls") pod "maas-api-e2e-mt-9e2107d7-84679db678-nvqjk" (UID: "b6aaaf38-664b-4581-b2dc-879ffc983d6c") : secret "maas-api-e2e-mt-9e2107d7-serving-cert" not found
Jul 30 21:05:23.203490 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.203470    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6aaaf38-664b-4581-b2dc-879ffc983d6c-openshift-service-ca\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.212229 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.212205    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5xdc\" (UniqueName: \"kubernetes.io/projected/b6aaaf38-664b-4581-b2dc-879ffc983d6c-kube-api-access-s5xdc\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.707474 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.707433    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6aaaf38-664b-4581-b2dc-879ffc983d6c-maas-api-tls\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.710102 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.710078    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6aaaf38-664b-4581-b2dc-879ffc983d6c-maas-api-tls\") pod \"maas-api-e2e-mt-9e2107d7-84679db678-nvqjk\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:23.782453 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.782424    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"]
Jul 30 21:05:23.785937 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.785920    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:05:23.788529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.788513    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9e2107d7\""
Jul 30 21:05:23.788610 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.788527    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9e2107d7-dockercfg-sjxhw\""
Jul 30 21:05:23.797061 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.797035    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"]
Jul 30 21:05:23.821673 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.821643    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"]
Jul 30 21:05:23.825625 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.825608    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:23.834474 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.834452    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"]
Jul 30 21:05:23.909233 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.909203    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cd17a2e-6458-4d8e-a87f-d621044670de-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq\" (UID: \"6cd17a2e-6458-4d8e-a87f-d621044670de\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:05:23.909233 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.909235    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/114668df-6164-4450-ad1d-9f01022a5fe1-plugins-config-volume\") pod \"payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn\" (UID: \"114668df-6164-4450-ad1d-9f01022a5fe1\") " pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:23.909401 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.909277    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjdnz\" (UniqueName: \"kubernetes.io/projected/6cd17a2e-6458-4d8e-a87f-d621044670de-kube-api-access-wjdnz\") pod \"payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq\" (UID: \"6cd17a2e-6458-4d8e-a87f-d621044670de\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:05:23.909401 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.909322    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmh4\" (UniqueName: \"kubernetes.io/projected/114668df-6164-4450-ad1d-9f01022a5fe1-kube-api-access-5pmh4\") pod \"payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn\" (UID: \"114668df-6164-4450-ad1d-9f01022a5fe1\") " pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:23.970842 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:23.970778    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:24.010229 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.010192    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cd17a2e-6458-4d8e-a87f-d621044670de-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq\" (UID: \"6cd17a2e-6458-4d8e-a87f-d621044670de\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:05:24.010372 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.010241    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/114668df-6164-4450-ad1d-9f01022a5fe1-plugins-config-volume\") pod \"payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn\" (UID: \"114668df-6164-4450-ad1d-9f01022a5fe1\") " pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:24.010372 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.010301    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjdnz\" (UniqueName: \"kubernetes.io/projected/6cd17a2e-6458-4d8e-a87f-d621044670de-kube-api-access-wjdnz\") pod \"payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq\" (UID: \"6cd17a2e-6458-4d8e-a87f-d621044670de\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:05:24.010372 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.010344    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmh4\" (UniqueName: \"kubernetes.io/projected/114668df-6164-4450-ad1d-9f01022a5fe1-kube-api-access-5pmh4\") pod \"payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn\" (UID: \"114668df-6164-4450-ad1d-9f01022a5fe1\") " pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:24.011006 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.010983    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cd17a2e-6458-4d8e-a87f-d621044670de-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq\" (UID: \"6cd17a2e-6458-4d8e-a87f-d621044670de\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:05:24.011098 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.011009    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/114668df-6164-4450-ad1d-9f01022a5fe1-plugins-config-volume\") pod \"payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn\" (UID: \"114668df-6164-4450-ad1d-9f01022a5fe1\") " pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:24.018873 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.018808    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjdnz\" (UniqueName: \"kubernetes.io/projected/6cd17a2e-6458-4d8e-a87f-d621044670de-kube-api-access-wjdnz\") pod \"payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq\" (UID: \"6cd17a2e-6458-4d8e-a87f-d621044670de\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:05:24.019297 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.019259    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmh4\" (UniqueName: \"kubernetes.io/projected/114668df-6164-4450-ad1d-9f01022a5fe1-kube-api-access-5pmh4\") pod \"payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn\" (UID: \"114668df-6164-4450-ad1d-9f01022a5fe1\") " pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:24.099155 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.099101    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:05:24.137601 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.137567    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:24.244175 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.244143    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"]
Jul 30 21:05:24.245805 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:05:24.245777    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd17a2e_6458_4d8e_a87f_d621044670de.slice/crio-d2d79238c7e2ee90439bb75f956b2436897c239bb9aa638d76598b4e9f54e48a WatchSource:0}: Error finding container d2d79238c7e2ee90439bb75f956b2436897c239bb9aa638d76598b4e9f54e48a: Status 404 returned error can't find the container with id d2d79238c7e2ee90439bb75f956b2436897c239bb9aa638d76598b4e9f54e48a
Jul 30 21:05:24.286895 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.286859    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"]
Jul 30 21:05:24.287530 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:05:24.287492    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114668df_6164_4450_ad1d_9f01022a5fe1.slice/crio-396d057e421fa9b3d478f86ef3a265f89444d0294aa92058d85aaad802e39a6d WatchSource:0}: Error finding container 396d057e421fa9b3d478f86ef3a265f89444d0294aa92058d85aaad802e39a6d: Status 404 returned error can't find the container with id 396d057e421fa9b3d478f86ef3a265f89444d0294aa92058d85aaad802e39a6d
Jul 30 21:05:24.311726 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.311674    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"]
Jul 30 21:05:24.313287 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:05:24.313261    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6aaaf38_664b_4581_b2dc_879ffc983d6c.slice/crio-8d55866922c26928030271f60fc1dd1a9fd06bf11b052c08e0fa1e83ed069f7c WatchSource:0}: Error finding container 8d55866922c26928030271f60fc1dd1a9fd06bf11b052c08e0fa1e83ed069f7c: Status 404 returned error can't find the container with id 8d55866922c26928030271f60fc1dd1a9fd06bf11b052c08e0fa1e83ed069f7c
Jul 30 21:05:24.314604 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.314588    2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 21:05:24.360740 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.360677    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn" event={"ID":"114668df-6164-4450-ad1d-9f01022a5fe1","Type":"ContainerStarted","Data":"bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa"}
Jul 30 21:05:24.360861 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.360769    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:24.360861 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.360796    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn" event={"ID":"114668df-6164-4450-ad1d-9f01022a5fe1","Type":"ContainerStarted","Data":"396d057e421fa9b3d478f86ef3a265f89444d0294aa92058d85aaad802e39a6d"}
Jul 30 21:05:24.361922 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.361891    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk" event={"ID":"b6aaaf38-664b-4581-b2dc-879ffc983d6c","Type":"ContainerStarted","Data":"8d55866922c26928030271f60fc1dd1a9fd06bf11b052c08e0fa1e83ed069f7c"}
Jul 30 21:05:24.363321 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.363302    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq" event={"ID":"6cd17a2e-6458-4d8e-a87f-d621044670de","Type":"ContainerStarted","Data":"52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad"}
Jul 30 21:05:24.363437 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.363335    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq" event={"ID":"6cd17a2e-6458-4d8e-a87f-d621044670de","Type":"ContainerStarted","Data":"d2d79238c7e2ee90439bb75f956b2436897c239bb9aa638d76598b4e9f54e48a"}
Jul 30 21:05:24.363497 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.363461    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:05:24.376132 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.376093    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn" podStartSLOduration=1.376079951 podStartE2EDuration="1.376079951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:05:24.374820044 +0000 UTC m=+2402.835778854" watchObservedRunningTime="2026-07-30 21:05:24.376079951 +0000 UTC m=+2402.837038762"
Jul 30 21:05:24.392354 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:24.392316    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq" podStartSLOduration=1.392302189 podStartE2EDuration="1.392302189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:05:24.390783096 +0000 UTC m=+2402.851741908" watchObservedRunningTime="2026-07-30 21:05:24.392302189 +0000 UTC m=+2402.853261000"
Jul 30 21:05:25.369726 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:25.369672    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk" event={"ID":"b6aaaf38-664b-4581-b2dc-879ffc983d6c","Type":"ContainerStarted","Data":"88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600"}
Jul 30 21:05:25.370110 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:25.369849    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:25.385719 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:25.385657    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk" podStartSLOduration=1.982763838 podStartE2EDuration="2.38564259s" totalImagesPullingTime="402.878752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:05:25.384313857 +0000 UTC m=+2403.845272669" watchObservedRunningTime="2026-07-30 21:05:25.38564259 +0000 UTC m=+2403.846601400"
Jul 30 21:05:31.380842 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:31.380815    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:05:35.371695 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:35.371666    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:05:35.372138 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:05:35.372087    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:06:47.286120 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.286089    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"]
Jul 30 21:06:47.286612 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.286285    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk" podUID="b6aaaf38-664b-4581-b2dc-879ffc983d6c" containerName="maas-api" containerID="cri-o://88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600" gracePeriod=30
Jul 30 21:06:47.370259 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.370230    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"]
Jul 30 21:06:47.370487 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.370455    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn" podUID="114668df-6164-4450-ad1d-9f01022a5fe1" containerName="payload-processing" containerID="cri-o://bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa" gracePeriod=30
Jul 30 21:06:47.378682 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.378656    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"]
Jul 30 21:06:47.378927 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.378896    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq" podUID="6cd17a2e-6458-4d8e-a87f-d621044670de" containerName="payload-pre-processing" containerID="cri-o://52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad" gracePeriod=30
Jul 30 21:06:47.650582 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.650554    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:06:47.653385 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.653357    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:06:47.675161 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.675142    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:06:47.749981 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.749955    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjdnz\" (UniqueName: \"kubernetes.io/projected/6cd17a2e-6458-4d8e-a87f-d621044670de-kube-api-access-wjdnz\") pod \"6cd17a2e-6458-4d8e-a87f-d621044670de\" (UID: \"6cd17a2e-6458-4d8e-a87f-d621044670de\") "
Jul 30 21:06:47.750148 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.750014    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pmh4\" (UniqueName: \"kubernetes.io/projected/114668df-6164-4450-ad1d-9f01022a5fe1-kube-api-access-5pmh4\") pod \"114668df-6164-4450-ad1d-9f01022a5fe1\" (UID: \"114668df-6164-4450-ad1d-9f01022a5fe1\") "
Jul 30 21:06:47.750148 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.750047    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6aaaf38-664b-4581-b2dc-879ffc983d6c-maas-api-tls\") pod \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") "
Jul 30 21:06:47.750148 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.750108    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5xdc\" (UniqueName: \"kubernetes.io/projected/b6aaaf38-664b-4581-b2dc-879ffc983d6c-kube-api-access-s5xdc\") pod \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") "
Jul 30 21:06:47.750148 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.750142    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/114668df-6164-4450-ad1d-9f01022a5fe1-plugins-config-volume\") pod \"114668df-6164-4450-ad1d-9f01022a5fe1\" (UID: \"114668df-6164-4450-ad1d-9f01022a5fe1\") "
Jul 30 21:06:47.750376 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.750198    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6aaaf38-664b-4581-b2dc-879ffc983d6c-openshift-service-ca\") pod \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\" (UID: \"b6aaaf38-664b-4581-b2dc-879ffc983d6c\") "
Jul 30 21:06:47.750376 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.750254    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cd17a2e-6458-4d8e-a87f-d621044670de-plugins-config-volume\") pod \"6cd17a2e-6458-4d8e-a87f-d621044670de\" (UID: \"6cd17a2e-6458-4d8e-a87f-d621044670de\") "
Jul 30 21:06:47.750668 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.750636    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/114668df-6164-4450-ad1d-9f01022a5fe1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "114668df-6164-4450-ad1d-9f01022a5fe1" (UID: "114668df-6164-4450-ad1d-9f01022a5fe1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:06:47.750788 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.750664    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6aaaf38-664b-4581-b2dc-879ffc983d6c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6aaaf38-664b-4581-b2dc-879ffc983d6c" (UID: "b6aaaf38-664b-4581-b2dc-879ffc983d6c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:06:47.751105 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.751077    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cd17a2e-6458-4d8e-a87f-d621044670de-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6cd17a2e-6458-4d8e-a87f-d621044670de" (UID: "6cd17a2e-6458-4d8e-a87f-d621044670de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:06:47.752388 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.752363    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6aaaf38-664b-4581-b2dc-879ffc983d6c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6aaaf38-664b-4581-b2dc-879ffc983d6c" (UID: "b6aaaf38-664b-4581-b2dc-879ffc983d6c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:06:47.752615 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.752596    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cd17a2e-6458-4d8e-a87f-d621044670de-kube-api-access-wjdnz" (OuterVolumeSpecName: "kube-api-access-wjdnz") pod "6cd17a2e-6458-4d8e-a87f-d621044670de" (UID: "6cd17a2e-6458-4d8e-a87f-d621044670de"). InnerVolumeSpecName "kube-api-access-wjdnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:06:47.753064 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.753044    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114668df-6164-4450-ad1d-9f01022a5fe1-kube-api-access-5pmh4" (OuterVolumeSpecName: "kube-api-access-5pmh4") pod "114668df-6164-4450-ad1d-9f01022a5fe1" (UID: "114668df-6164-4450-ad1d-9f01022a5fe1"). InnerVolumeSpecName "kube-api-access-5pmh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:06:47.753163 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.753133    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6aaaf38-664b-4581-b2dc-879ffc983d6c-kube-api-access-s5xdc" (OuterVolumeSpecName: "kube-api-access-s5xdc") pod "b6aaaf38-664b-4581-b2dc-879ffc983d6c" (UID: "b6aaaf38-664b-4581-b2dc-879ffc983d6c"). InnerVolumeSpecName "kube-api-access-s5xdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:06:47.782111 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.782083    2547 generic.go:356] "Generic (PLEG): container finished" podID="6cd17a2e-6458-4d8e-a87f-d621044670de" containerID="52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad" exitCode=0
Jul 30 21:06:47.782323 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.782140    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"
Jul 30 21:06:47.782323 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.782173    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq" event={"ID":"6cd17a2e-6458-4d8e-a87f-d621044670de","Type":"ContainerDied","Data":"52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad"}
Jul 30 21:06:47.782323 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.782247    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq" event={"ID":"6cd17a2e-6458-4d8e-a87f-d621044670de","Type":"ContainerDied","Data":"d2d79238c7e2ee90439bb75f956b2436897c239bb9aa638d76598b4e9f54e48a"}
Jul 30 21:06:47.782323 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.782280    2547 scope.go:117] "RemoveContainer" containerID="52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad"
Jul 30 21:06:47.783811 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.783730    2547 generic.go:356] "Generic (PLEG): container finished" podID="114668df-6164-4450-ad1d-9f01022a5fe1" containerID="bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa" exitCode=0
Jul 30 21:06:47.783811 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.783758    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn" event={"ID":"114668df-6164-4450-ad1d-9f01022a5fe1","Type":"ContainerDied","Data":"bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa"}
Jul 30 21:06:47.783952 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.783825    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn" event={"ID":"114668df-6164-4450-ad1d-9f01022a5fe1","Type":"ContainerDied","Data":"396d057e421fa9b3d478f86ef3a265f89444d0294aa92058d85aaad802e39a6d"}
Jul 30 21:06:47.783952 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.783775    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"
Jul 30 21:06:47.786292 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.785392    2547 generic.go:356] "Generic (PLEG): container finished" podID="b6aaaf38-664b-4581-b2dc-879ffc983d6c" containerID="88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600" exitCode=0
Jul 30 21:06:47.786292 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.785423    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk" event={"ID":"b6aaaf38-664b-4581-b2dc-879ffc983d6c","Type":"ContainerDied","Data":"88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600"}
Jul 30 21:06:47.786292 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.785452    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"
Jul 30 21:06:47.786292 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.785457    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk" event={"ID":"b6aaaf38-664b-4581-b2dc-879ffc983d6c","Type":"ContainerDied","Data":"8d55866922c26928030271f60fc1dd1a9fd06bf11b052c08e0fa1e83ed069f7c"}
Jul 30 21:06:47.803581 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.803542    2547 scope.go:117] "RemoveContainer" containerID="52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad"
Jul 30 21:06:47.804217 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:06:47.804168    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad\": container with ID starting with 52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad not found: ID does not exist" containerID="52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad"
Jul 30 21:06:47.804729 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.804675    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad"} err="failed to get container status \"52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad\": rpc error: code = NotFound desc = could not find container \"52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad\": container with ID starting with 52b88f5cb9cd0c723e2ed55b2d2268a08b0c48c21c5b62af533ce8b6fed9dbad not found: ID does not exist"
Jul 30 21:06:47.804862 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.804846    2547 scope.go:117] "RemoveContainer" containerID="bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa"
Jul 30 21:06:47.813923 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.813904    2547 scope.go:117] "RemoveContainer" containerID="bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa"
Jul 30 21:06:47.814197 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:06:47.814170    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa\": container with ID starting with bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa not found: ID does not exist" containerID="bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa"
Jul 30 21:06:47.814290 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.814205    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa"} err="failed to get container status \"bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa\": rpc error: code = NotFound desc = could not find container \"bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa\": container with ID starting with bc463c045a554114cacfb213346e1033da33547deb73c63a2a1f3b23083b2daa not found: ID does not exist"
Jul 30 21:06:47.814290 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.814227    2547 scope.go:117] "RemoveContainer" containerID="88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600"
Jul 30 21:06:47.816313 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.816176    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"]
Jul 30 21:06:47.819544 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.819523    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9e2107d7-5c5cb9d648-6fqqq"]
Jul 30 21:06:47.822586 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.822572    2547 scope.go:117] "RemoveContainer" containerID="88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600"
Jul 30 21:06:47.822906 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:06:47.822885    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600\": container with ID starting with 88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600 not found: ID does not exist" containerID="88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600"
Jul 30 21:06:47.822974 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.822914    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600"} err="failed to get container status \"88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600\": rpc error: code = NotFound desc = could not find container \"88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600\": container with ID starting with 88f21aa71c43f50f94beb2f3a2bb60e87af7a00980938ee1f3957e65c92b3600 not found: ID does not exist"
Jul 30 21:06:47.833517 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.833496    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"]
Jul 30 21:06:47.836610 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.836588    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9e2107d7-559dd565db-dmffn"]
Jul 30 21:06:47.848136 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.848114    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"]
Jul 30 21:06:47.851204 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.851181    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjdnz\" (UniqueName: \"kubernetes.io/projected/6cd17a2e-6458-4d8e-a87f-d621044670de-kube-api-access-wjdnz\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:06:47.851204 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.851206    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pmh4\" (UniqueName: \"kubernetes.io/projected/114668df-6164-4450-ad1d-9f01022a5fe1-kube-api-access-5pmh4\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:06:47.851364 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.851216    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6aaaf38-664b-4581-b2dc-879ffc983d6c-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:06:47.851364 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.851226    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5xdc\" (UniqueName: \"kubernetes.io/projected/b6aaaf38-664b-4581-b2dc-879ffc983d6c-kube-api-access-s5xdc\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:06:47.851364 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.851233    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/114668df-6164-4450-ad1d-9f01022a5fe1-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:06:47.851364 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.851242    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6aaaf38-664b-4581-b2dc-879ffc983d6c-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:06:47.851364 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.851251    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cd17a2e-6458-4d8e-a87f-d621044670de-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:06:47.852675 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:47.852655    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9e2107d7-84679db678-nvqjk"]
Jul 30 21:06:48.072668 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:48.072645    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="114668df-6164-4450-ad1d-9f01022a5fe1" path="/var/lib/kubelet/pods/114668df-6164-4450-ad1d-9f01022a5fe1/volumes"
Jul 30 21:06:48.073016 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:48.073002    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cd17a2e-6458-4d8e-a87f-d621044670de" path="/var/lib/kubelet/pods/6cd17a2e-6458-4d8e-a87f-d621044670de/volumes"
Jul 30 21:06:48.073337 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:06:48.073325    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6aaaf38-664b-4581-b2dc-879ffc983d6c" path="/var/lib/kubelet/pods/b6aaaf38-664b-4581-b2dc-879ffc983d6c/volumes"
Jul 30 21:09:03.104893 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.104808    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"]
Jul 30 21:09:03.105440 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.105364    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cd17a2e-6458-4d8e-a87f-d621044670de" containerName="payload-pre-processing"
Jul 30 21:09:03.105440 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.105384    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cd17a2e-6458-4d8e-a87f-d621044670de" containerName="payload-pre-processing"
Jul 30 21:09:03.105440 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.105398    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114668df-6164-4450-ad1d-9f01022a5fe1" containerName="payload-processing"
Jul 30 21:09:03.105440 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.105407    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="114668df-6164-4450-ad1d-9f01022a5fe1" containerName="payload-processing"
Jul 30 21:09:03.105440 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.105435    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6aaaf38-664b-4581-b2dc-879ffc983d6c" containerName="maas-api"
Jul 30 21:09:03.105440 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.105443    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6aaaf38-664b-4581-b2dc-879ffc983d6c" containerName="maas-api"
Jul 30 21:09:03.105790 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.105521    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6aaaf38-664b-4581-b2dc-879ffc983d6c" containerName="maas-api"
Jul 30 21:09:03.105790 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.105531    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cd17a2e-6458-4d8e-a87f-d621044670de" containerName="payload-pre-processing"
Jul 30 21:09:03.105790 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.105543    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="114668df-6164-4450-ad1d-9f01022a5fe1" containerName="payload-processing"
Jul 30 21:09:03.108973 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.108950    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.111421 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.111402    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-d59412-serving-cert\""
Jul 30 21:09:03.118020 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.118000    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"]
Jul 30 21:09:03.237290 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.237256    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-maas-api-tls\") pod \"maas-api-e2e-inf-a-d59412-665bf9c78-25pjx\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.237476 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.237304    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d59412-665bf9c78-25pjx\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.237476 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.237408    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qldzb\" (UniqueName: \"kubernetes.io/projected/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-kube-api-access-qldzb\") pod \"maas-api-e2e-inf-a-d59412-665bf9c78-25pjx\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.338547 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.338517    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-maas-api-tls\") pod \"maas-api-e2e-inf-a-d59412-665bf9c78-25pjx\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.338718 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.338566    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d59412-665bf9c78-25pjx\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.338718 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.338606    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qldzb\" (UniqueName: \"kubernetes.io/projected/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-kube-api-access-qldzb\") pod \"maas-api-e2e-inf-a-d59412-665bf9c78-25pjx\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.339339 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.339234    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d59412-665bf9c78-25pjx\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.341015 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.340992    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-maas-api-tls\") pod \"maas-api-e2e-inf-a-d59412-665bf9c78-25pjx\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.346768 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.346746    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qldzb\" (UniqueName: \"kubernetes.io/projected/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-kube-api-access-qldzb\") pod \"maas-api-e2e-inf-a-d59412-665bf9c78-25pjx\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.421955 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.421835    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:03.560993 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.560963    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"]
Jul 30 21:09:03.562775 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:09:03.562746    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcecbf90b_1d6a_4a16_9d0d_10767b6ae4e2.slice/crio-5c750237bfe1a0f56a643ef0d3918cc2a3119c6bb1423058a8fc4568c5ca9957 WatchSource:0}: Error finding container 5c750237bfe1a0f56a643ef0d3918cc2a3119c6bb1423058a8fc4568c5ca9957: Status 404 returned error can't find the container with id 5c750237bfe1a0f56a643ef0d3918cc2a3119c6bb1423058a8fc4568c5ca9957
Jul 30 21:09:03.730213 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.730127    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"]
Jul 30 21:09:03.734731 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.734685    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:03.737557 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.737532    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-d59412\""
Jul 30 21:09:03.737683 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.737637    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-d59412-dockercfg-kbh2g\""
Jul 30 21:09:03.745245 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.745221    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"]
Jul 30 21:09:03.767571 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.767543    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"]
Jul 30 21:09:03.771034 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.771016    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:03.779593 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.779570    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"]
Jul 30 21:09:03.842869 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.842839    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abf8eab-9386-4490-887b-2bb656be9dd9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6\" (UID: \"0abf8eab-9386-4490-887b-2bb656be9dd9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:03.842869 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.842870    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpnfs\" (UniqueName: \"kubernetes.io/projected/0abf8eab-9386-4490-887b-2bb656be9dd9-kube-api-access-wpnfs\") pod \"payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6\" (UID: \"0abf8eab-9386-4490-887b-2bb656be9dd9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:03.843061 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.842933    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df87db58-e01c-4e5f-8b1f-695db4352ab6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v\" (UID: \"df87db58-e01c-4e5f-8b1f-695db4352ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:03.843099 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.843066    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvtgb\" (UniqueName: \"kubernetes.io/projected/df87db58-e01c-4e5f-8b1f-695db4352ab6-kube-api-access-mvtgb\") pod \"payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v\" (UID: \"df87db58-e01c-4e5f-8b1f-695db4352ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:03.943666 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.943633    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df87db58-e01c-4e5f-8b1f-695db4352ab6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v\" (UID: \"df87db58-e01c-4e5f-8b1f-695db4352ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:03.943854 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.943729    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvtgb\" (UniqueName: \"kubernetes.io/projected/df87db58-e01c-4e5f-8b1f-695db4352ab6-kube-api-access-mvtgb\") pod \"payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v\" (UID: \"df87db58-e01c-4e5f-8b1f-695db4352ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:03.943854 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.943781    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abf8eab-9386-4490-887b-2bb656be9dd9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6\" (UID: \"0abf8eab-9386-4490-887b-2bb656be9dd9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:03.943854 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.943813    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpnfs\" (UniqueName: \"kubernetes.io/projected/0abf8eab-9386-4490-887b-2bb656be9dd9-kube-api-access-wpnfs\") pod \"payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6\" (UID: \"0abf8eab-9386-4490-887b-2bb656be9dd9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:03.944326 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.944306    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df87db58-e01c-4e5f-8b1f-695db4352ab6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v\" (UID: \"df87db58-e01c-4e5f-8b1f-695db4352ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:03.944436 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.944419    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abf8eab-9386-4490-887b-2bb656be9dd9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6\" (UID: \"0abf8eab-9386-4490-887b-2bb656be9dd9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:03.954795 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.954768    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvtgb\" (UniqueName: \"kubernetes.io/projected/df87db58-e01c-4e5f-8b1f-695db4352ab6-kube-api-access-mvtgb\") pod \"payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v\" (UID: \"df87db58-e01c-4e5f-8b1f-695db4352ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:03.954895 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:03.954817    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpnfs\" (UniqueName: \"kubernetes.io/projected/0abf8eab-9386-4490-887b-2bb656be9dd9-kube-api-access-wpnfs\") pod \"payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6\" (UID: \"0abf8eab-9386-4490-887b-2bb656be9dd9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:04.045626 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.045563    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:04.082419 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.082389    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:04.204216 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.204172    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"]
Jul 30 21:09:04.205552 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:09:04.205493    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf87db58_e01c_4e5f_8b1f_695db4352ab6.slice/crio-e3730043cfb393fc86f8604b4a2bd89b752f1a6e99ec946a9b2cfb6dad812ccf WatchSource:0}: Error finding container e3730043cfb393fc86f8604b4a2bd89b752f1a6e99ec946a9b2cfb6dad812ccf: Status 404 returned error can't find the container with id e3730043cfb393fc86f8604b4a2bd89b752f1a6e99ec946a9b2cfb6dad812ccf
Jul 30 21:09:04.238470 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.238428    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"]
Jul 30 21:09:04.240871 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:09:04.240849    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0abf8eab_9386_4490_887b_2bb656be9dd9.slice/crio-6141593007f17c86c9eb1f4f447b174e50e10a705b03f7122ba5391d9bdfa67d WatchSource:0}: Error finding container 6141593007f17c86c9eb1f4f447b174e50e10a705b03f7122ba5391d9bdfa67d: Status 404 returned error can't find the container with id 6141593007f17c86c9eb1f4f447b174e50e10a705b03f7122ba5391d9bdfa67d
Jul 30 21:09:04.432485 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.432448    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6" event={"ID":"0abf8eab-9386-4490-887b-2bb656be9dd9","Type":"ContainerStarted","Data":"4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394"}
Jul 30 21:09:04.432648 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.432509    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6" event={"ID":"0abf8eab-9386-4490-887b-2bb656be9dd9","Type":"ContainerStarted","Data":"6141593007f17c86c9eb1f4f447b174e50e10a705b03f7122ba5391d9bdfa67d"}
Jul 30 21:09:04.432648 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.432571    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:04.433982 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.433960    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx" event={"ID":"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2","Type":"ContainerStarted","Data":"d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1"}
Jul 30 21:09:04.434216 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.434000    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx" event={"ID":"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2","Type":"ContainerStarted","Data":"5c750237bfe1a0f56a643ef0d3918cc2a3119c6bb1423058a8fc4568c5ca9957"}
Jul 30 21:09:04.434216 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.434028    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:04.435495 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.435473    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v" event={"ID":"df87db58-e01c-4e5f-8b1f-695db4352ab6","Type":"ContainerStarted","Data":"9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce"}
Jul 30 21:09:04.435599 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.435507    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v" event={"ID":"df87db58-e01c-4e5f-8b1f-695db4352ab6","Type":"ContainerStarted","Data":"e3730043cfb393fc86f8604b4a2bd89b752f1a6e99ec946a9b2cfb6dad812ccf"}
Jul 30 21:09:04.435649 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.435616    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:04.448895 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.448855    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6" podStartSLOduration=1.4488435370000001 podStartE2EDuration="1.448843537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:09:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:09:04.447289229 +0000 UTC m=+2622.908248044" watchObservedRunningTime="2026-07-30 21:09:04.448843537 +0000 UTC m=+2622.909802347"
Jul 30 21:09:04.461138 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.461103    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v" podStartSLOduration=1.461089974 podStartE2EDuration="1.461089974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:09:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:09:04.460729655 +0000 UTC m=+2622.921688468" watchObservedRunningTime="2026-07-30 21:09:04.461089974 +0000 UTC m=+2622.922048796"
Jul 30 21:09:04.477207 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:04.477167    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx" podStartSLOduration=0.986353407 podStartE2EDuration="1.477155912s" totalImagesPullingTime="490.802505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:09:04.475412429 +0000 UTC m=+2622.936371242" watchObservedRunningTime="2026-07-30 21:09:04.477155912 +0000 UTC m=+2622.938114726"
Jul 30 21:09:10.445995 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:10.445963    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:09:15.442478 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:15.442448    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:09:15.442955 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:15.442520    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:09:31.243291 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.243261    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"]
Jul 30 21:09:31.247117 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.247097    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.249651 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.249629    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3230ac-serving-cert\""
Jul 30 21:09:31.259093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.259074    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"]
Jul 30 21:09:31.352168 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.352143    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4rkp\" (UniqueName: \"kubernetes.io/projected/76e737cf-579b-4ac5-8a85-ca597f7d3b21-kube-api-access-g4rkp\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.352277 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.352202    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76e737cf-579b-4ac5-8a85-ca597f7d3b21-maas-api-tls\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.352324 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.352288    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76e737cf-579b-4ac5-8a85-ca597f7d3b21-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.452793 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.452764    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76e737cf-579b-4ac5-8a85-ca597f7d3b21-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.452903 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.452831    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rkp\" (UniqueName: \"kubernetes.io/projected/76e737cf-579b-4ac5-8a85-ca597f7d3b21-kube-api-access-g4rkp\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.452903 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.452894    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76e737cf-579b-4ac5-8a85-ca597f7d3b21-maas-api-tls\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.453023 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:09:31.453008    2547 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-serving-cert: secret "maas-api-e2e-inf-b-3230ac-serving-cert" not found
Jul 30 21:09:31.453098 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:09:31.453086    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76e737cf-579b-4ac5-8a85-ca597f7d3b21-maas-api-tls podName:76e737cf-579b-4ac5-8a85-ca597f7d3b21 nodeName:}" failed. No retries permitted until 2026-07-30 21:09:31.9530662 +0000 UTC m=+2650.414025004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/76e737cf-579b-4ac5-8a85-ca597f7d3b21-maas-api-tls") pod "maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv" (UID: "76e737cf-579b-4ac5-8a85-ca597f7d3b21") : secret "maas-api-e2e-inf-b-3230ac-serving-cert" not found
Jul 30 21:09:31.453309 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.453289    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76e737cf-579b-4ac5-8a85-ca597f7d3b21-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.461125 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.461109    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4rkp\" (UniqueName: \"kubernetes.io/projected/76e737cf-579b-4ac5-8a85-ca597f7d3b21-kube-api-access-g4rkp\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.867660 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.867620    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"]
Jul 30 21:09:31.871251 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.871235    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:09:31.873708 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.873684    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-3230ac\""
Jul 30 21:09:31.873852 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.873833    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-3230ac-dockercfg-shrfh\""
Jul 30 21:09:31.880689 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.880664    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"]
Jul 30 21:09:31.889960 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.889940    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"]
Jul 30 21:09:31.893402 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.893383    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:31.903918 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.903896    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"]
Jul 30 21:09:31.956361 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.956333    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6474b\" (UniqueName: \"kubernetes.io/projected/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-kube-api-access-6474b\") pod \"payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72\" (UID: \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:09:31.956463 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.956392    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc8f1782-30d3-44de-a39d-0771b86dbe42-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb\" (UID: \"bc8f1782-30d3-44de-a39d-0771b86dbe42\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:31.956512 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.956465    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kszwq\" (UniqueName: \"kubernetes.io/projected/bc8f1782-30d3-44de-a39d-0771b86dbe42-kube-api-access-kszwq\") pod \"payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb\" (UID: \"bc8f1782-30d3-44de-a39d-0771b86dbe42\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:31.956557 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.956509    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76e737cf-579b-4ac5-8a85-ca597f7d3b21-maas-api-tls\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:31.956557 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.956527    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72\" (UID: \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:09:31.958837 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:31.958822    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76e737cf-579b-4ac5-8a85-ca597f7d3b21-maas-api-tls\") pod \"maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:32.057211 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.057179    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc8f1782-30d3-44de-a39d-0771b86dbe42-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb\" (UID: \"bc8f1782-30d3-44de-a39d-0771b86dbe42\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:32.057353 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.057231    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszwq\" (UniqueName: \"kubernetes.io/projected/bc8f1782-30d3-44de-a39d-0771b86dbe42-kube-api-access-kszwq\") pod \"payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb\" (UID: \"bc8f1782-30d3-44de-a39d-0771b86dbe42\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:32.057432 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.057406    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72\" (UID: \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:09:32.057503 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.057485    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6474b\" (UniqueName: \"kubernetes.io/projected/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-kube-api-access-6474b\") pod \"payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72\" (UID: \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:09:32.057979 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.057956    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc8f1782-30d3-44de-a39d-0771b86dbe42-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb\" (UID: \"bc8f1782-30d3-44de-a39d-0771b86dbe42\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:32.058135 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.058117    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72\" (UID: \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:09:32.066035 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.066014    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kszwq\" (UniqueName: \"kubernetes.io/projected/bc8f1782-30d3-44de-a39d-0771b86dbe42-kube-api-access-kszwq\") pod \"payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb\" (UID: \"bc8f1782-30d3-44de-a39d-0771b86dbe42\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:32.066285 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.066268    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6474b\" (UniqueName: \"kubernetes.io/projected/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-kube-api-access-6474b\") pod \"payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72\" (UID: \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:09:32.158141 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.158078    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:32.182927 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.182900    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:09:32.206067 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.206039    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:32.323884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.323806    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"]
Jul 30 21:09:32.325871 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:09:32.325665    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76e737cf_579b_4ac5_8a85_ca597f7d3b21.slice/crio-90ab77c9b008d5eff03e61673e49fba535a4b569649ff4f14d4a85cf5e114682 WatchSource:0}: Error finding container 90ab77c9b008d5eff03e61673e49fba535a4b569649ff4f14d4a85cf5e114682: Status 404 returned error can't find the container with id 90ab77c9b008d5eff03e61673e49fba535a4b569649ff4f14d4a85cf5e114682
Jul 30 21:09:32.376570 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.376546    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"]
Jul 30 21:09:32.378143 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:09:32.378118    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc8f1782_30d3_44de_a39d_0771b86dbe42.slice/crio-28bdeb6cddc569da54ddf3a643fba6b85ca1928e5e7f6a168338587141c53b35 WatchSource:0}: Error finding container 28bdeb6cddc569da54ddf3a643fba6b85ca1928e5e7f6a168338587141c53b35: Status 404 returned error can't find the container with id 28bdeb6cddc569da54ddf3a643fba6b85ca1928e5e7f6a168338587141c53b35
Jul 30 21:09:32.557316 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.557289    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"]
Jul 30 21:09:32.558978 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:09:32.558951    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75d7a5c7_3bc7_4855_95fe_5057c6bf79ff.slice/crio-957bc6fb6777ea47f349067415a706234e86fda41c0b5aa0101f90dac42acf17 WatchSource:0}: Error finding container 957bc6fb6777ea47f349067415a706234e86fda41c0b5aa0101f90dac42acf17: Status 404 returned error can't find the container with id 957bc6fb6777ea47f349067415a706234e86fda41c0b5aa0101f90dac42acf17
Jul 30 21:09:32.573288 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.573257    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb" event={"ID":"bc8f1782-30d3-44de-a39d-0771b86dbe42","Type":"ContainerStarted","Data":"945ccf25ae4b982bc324279e143bc705288cd1b988e10ca6a7e5a5c346e90b45"}
Jul 30 21:09:32.573416 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.573320    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb" event={"ID":"bc8f1782-30d3-44de-a39d-0771b86dbe42","Type":"ContainerStarted","Data":"28bdeb6cddc569da54ddf3a643fba6b85ca1928e5e7f6a168338587141c53b35"}
Jul 30 21:09:32.573416 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.573366    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:32.574433 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.574414    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv" event={"ID":"76e737cf-579b-4ac5-8a85-ca597f7d3b21","Type":"ContainerStarted","Data":"90ab77c9b008d5eff03e61673e49fba535a4b569649ff4f14d4a85cf5e114682"}
Jul 30 21:09:32.575632 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.575615    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72" event={"ID":"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff","Type":"ContainerStarted","Data":"957bc6fb6777ea47f349067415a706234e86fda41c0b5aa0101f90dac42acf17"}
Jul 30 21:09:32.588299 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:32.588259    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb" podStartSLOduration=1.588246762 podStartE2EDuration="1.588246762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:09:32.588042383 +0000 UTC m=+2651.049001195" watchObservedRunningTime="2026-07-30 21:09:32.588246762 +0000 UTC m=+2651.049205568"
Jul 30 21:09:33.582538 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:33.582511    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72" event={"ID":"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff","Type":"ContainerStarted","Data":"551b2b21b0d8dc2a00227523e9f17ce829378fac2694da04ae618f2b39b2d998"}
Jul 30 21:09:33.582965 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:33.582587    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:09:33.598353 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:33.598305    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72" podStartSLOduration=2.598288213 podStartE2EDuration="2.598288213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:09:33.597591534 +0000 UTC m=+2652.058550345" watchObservedRunningTime="2026-07-30 21:09:33.598288213 +0000 UTC m=+2652.059247015"
Jul 30 21:09:34.591739 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:34.591679    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv" event={"ID":"76e737cf-579b-4ac5-8a85-ca597f7d3b21","Type":"ContainerStarted","Data":"2b5cce8baa4abf67dbdb5815a0f9db47870b1c867a21918f7822f593ef805c76"}
Jul 30 21:09:34.610269 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:34.610201    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv" podStartSLOduration=2.353202585 podStartE2EDuration="3.61018357s" totalImagesPullingTime="1.256980985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:09:34.610127769 +0000 UTC m=+2653.071086606" watchObservedRunningTime="2026-07-30 21:09:34.61018357 +0000 UTC m=+2653.071142380"
Jul 30 21:09:35.597626 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:35.597592    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:41.608055 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:41.608026    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:09:43.584618 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:43.584583    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:09:44.593858 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:09:44.593830    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:10:22.224584 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:10:22.224557    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:10:22.228655 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:10:22.228634    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:10:22.234319 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:10:22.234297    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:10:22.237908 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:10:22.237892    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:11:26.633952 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:26.633911    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"]
Jul 30 21:11:26.634460 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:26.634167    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx" podUID="cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2" containerName="maas-api" containerID="cri-o://d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1" gracePeriod=30
Jul 30 21:11:26.691770 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:26.691740    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"]
Jul 30 21:11:26.691992 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:26.691948    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6" podUID="0abf8eab-9386-4490-887b-2bb656be9dd9" containerName="payload-processing" containerID="cri-o://4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394" gracePeriod=30
Jul 30 21:11:26.699882 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:26.699342    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"]
Jul 30 21:11:26.699882 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:26.699563    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v" podUID="df87db58-e01c-4e5f-8b1f-695db4352ab6" containerName="payload-pre-processing" containerID="cri-o://9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce" gracePeriod=30
Jul 30 21:11:26.987989 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:26.987963    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:11:26.989868 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:26.989844    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:11:27.028352 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.028322    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df87db58-e01c-4e5f-8b1f-695db4352ab6-plugins-config-volume\") pod \"df87db58-e01c-4e5f-8b1f-695db4352ab6\" (UID: \"df87db58-e01c-4e5f-8b1f-695db4352ab6\") "
Jul 30 21:11:27.028500 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.028407    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvtgb\" (UniqueName: \"kubernetes.io/projected/df87db58-e01c-4e5f-8b1f-695db4352ab6-kube-api-access-mvtgb\") pod \"df87db58-e01c-4e5f-8b1f-695db4352ab6\" (UID: \"df87db58-e01c-4e5f-8b1f-695db4352ab6\") "
Jul 30 21:11:27.028573 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.028522    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpnfs\" (UniqueName: \"kubernetes.io/projected/0abf8eab-9386-4490-887b-2bb656be9dd9-kube-api-access-wpnfs\") pod \"0abf8eab-9386-4490-887b-2bb656be9dd9\" (UID: \"0abf8eab-9386-4490-887b-2bb656be9dd9\") "
Jul 30 21:11:27.028611 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.028581    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abf8eab-9386-4490-887b-2bb656be9dd9-plugins-config-volume\") pod \"0abf8eab-9386-4490-887b-2bb656be9dd9\" (UID: \"0abf8eab-9386-4490-887b-2bb656be9dd9\") "
Jul 30 21:11:27.028731 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.028683    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df87db58-e01c-4e5f-8b1f-695db4352ab6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df87db58-e01c-4e5f-8b1f-695db4352ab6" (UID: "df87db58-e01c-4e5f-8b1f-695db4352ab6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:11:27.029287 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.028880    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df87db58-e01c-4e5f-8b1f-695db4352ab6-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:27.029287 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.029231    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0abf8eab-9386-4490-887b-2bb656be9dd9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0abf8eab-9386-4490-887b-2bb656be9dd9" (UID: "0abf8eab-9386-4490-887b-2bb656be9dd9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:11:27.031916 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.031690    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df87db58-e01c-4e5f-8b1f-695db4352ab6-kube-api-access-mvtgb" (OuterVolumeSpecName: "kube-api-access-mvtgb") pod "df87db58-e01c-4e5f-8b1f-695db4352ab6" (UID: "df87db58-e01c-4e5f-8b1f-695db4352ab6"). InnerVolumeSpecName "kube-api-access-mvtgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:11:27.032106 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.032082    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0abf8eab-9386-4490-887b-2bb656be9dd9-kube-api-access-wpnfs" (OuterVolumeSpecName: "kube-api-access-wpnfs") pod "0abf8eab-9386-4490-887b-2bb656be9dd9" (UID: "0abf8eab-9386-4490-887b-2bb656be9dd9"). InnerVolumeSpecName "kube-api-access-wpnfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:11:27.083298 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.083275    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:11:27.129877 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.129844    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-maas-api-tls\") pod \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") "
Jul 30 21:11:27.130016 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.129906    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-openshift-service-ca\") pod \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") "
Jul 30 21:11:27.130087 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.130048    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qldzb\" (UniqueName: \"kubernetes.io/projected/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-kube-api-access-qldzb\") pod \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\" (UID: \"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2\") "
Jul 30 21:11:27.130263 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.130242    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2" (UID: "cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:11:27.130447 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.130420    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpnfs\" (UniqueName: \"kubernetes.io/projected/0abf8eab-9386-4490-887b-2bb656be9dd9-kube-api-access-wpnfs\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:27.130447 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.130442    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abf8eab-9386-4490-887b-2bb656be9dd9-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:27.130592 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.130455    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvtgb\" (UniqueName: \"kubernetes.io/projected/df87db58-e01c-4e5f-8b1f-695db4352ab6-kube-api-access-mvtgb\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:27.130592 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.130469    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:27.132207 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.132179    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2" (UID: "cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:11:27.132507 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.132486    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-kube-api-access-qldzb" (OuterVolumeSpecName: "kube-api-access-qldzb") pod "cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2" (UID: "cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2"). InnerVolumeSpecName "kube-api-access-qldzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:11:27.197559 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.197495    2547 generic.go:356] "Generic (PLEG): container finished" podID="0abf8eab-9386-4490-887b-2bb656be9dd9" containerID="4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394" exitCode=0
Jul 30 21:11:27.197559 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.197545    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"
Jul 30 21:11:27.197745 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.197576    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6" event={"ID":"0abf8eab-9386-4490-887b-2bb656be9dd9","Type":"ContainerDied","Data":"4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394"}
Jul 30 21:11:27.197745 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.197646    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6" event={"ID":"0abf8eab-9386-4490-887b-2bb656be9dd9","Type":"ContainerDied","Data":"6141593007f17c86c9eb1f4f447b174e50e10a705b03f7122ba5391d9bdfa67d"}
Jul 30 21:11:27.197745 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.197680    2547 scope.go:117] "RemoveContainer" containerID="4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394"
Jul 30 21:11:27.199031 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.199007    2547 generic.go:356] "Generic (PLEG): container finished" podID="cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2" containerID="d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1" exitCode=0
Jul 30 21:11:27.199031 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.199034    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx" event={"ID":"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2","Type":"ContainerDied","Data":"d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1"}
Jul 30 21:11:27.199223 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.199061    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"
Jul 30 21:11:27.199223 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.199079    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx" event={"ID":"cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2","Type":"ContainerDied","Data":"5c750237bfe1a0f56a643ef0d3918cc2a3119c6bb1423058a8fc4568c5ca9957"}
Jul 30 21:11:27.201040 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.201017    2547 generic.go:356] "Generic (PLEG): container finished" podID="df87db58-e01c-4e5f-8b1f-695db4352ab6" containerID="9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce" exitCode=0
Jul 30 21:11:27.201169 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.201152    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v" event={"ID":"df87db58-e01c-4e5f-8b1f-695db4352ab6","Type":"ContainerDied","Data":"9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce"}
Jul 30 21:11:27.201224 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.201197    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v" event={"ID":"df87db58-e01c-4e5f-8b1f-695db4352ab6","Type":"ContainerDied","Data":"e3730043cfb393fc86f8604b4a2bd89b752f1a6e99ec946a9b2cfb6dad812ccf"}
Jul 30 21:11:27.201279 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.201258    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"
Jul 30 21:11:27.211852 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.211827    2547 scope.go:117] "RemoveContainer" containerID="4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394"
Jul 30 21:11:27.214931 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:11:27.214887    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394\": container with ID starting with 4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394 not found: ID does not exist" containerID="4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394"
Jul 30 21:11:27.215126 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.214929    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394"} err="failed to get container status \"4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394\": rpc error: code = NotFound desc = could not find container \"4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394\": container with ID starting with 4ff110c9947e7102e5faccd8d81dca429e3eef2fbdec6cd18e5236a882c2e394 not found: ID does not exist"
Jul 30 21:11:27.215126 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.214956    2547 scope.go:117] "RemoveContainer" containerID="d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1"
Jul 30 21:11:27.225926 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.225904    2547 scope.go:117] "RemoveContainer" containerID="d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1"
Jul 30 21:11:27.226169 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:11:27.226153    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1\": container with ID starting with d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1 not found: ID does not exist" containerID="d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1"
Jul 30 21:11:27.226228 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.226175    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1"} err="failed to get container status \"d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1\": rpc error: code = NotFound desc = could not find container \"d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1\": container with ID starting with d6c8339f8f6ba64500d034ac43d8df44262f9beffc65afef4fe3a49e3cbe44d1 not found: ID does not exist"
Jul 30 21:11:27.226228 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.226189    2547 scope.go:117] "RemoveContainer" containerID="9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce"
Jul 30 21:11:27.231073 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.231050    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:27.231142 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.231081    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qldzb\" (UniqueName: \"kubernetes.io/projected/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2-kube-api-access-qldzb\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:27.231641 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.231608    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"]
Jul 30 21:11:27.235160 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.235139    2547 scope.go:117] "RemoveContainer" containerID="9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce"
Jul 30 21:11:27.235228 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.235206    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d59412-665bf9c78-25pjx"]
Jul 30 21:11:27.235408 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:11:27.235390    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce\": container with ID starting with 9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce not found: ID does not exist" containerID="9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce"
Jul 30 21:11:27.235457 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.235414    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce"} err="failed to get container status \"9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce\": rpc error: code = NotFound desc = could not find container \"9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce\": container with ID starting with 9631c4e70d01aa792a5db209903cc6f23aec282358da25cc5c4580fa35be94ce not found: ID does not exist"
Jul 30 21:11:27.245922 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.245900    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"]
Jul 30 21:11:27.249857 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.249838    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d59412-868df79576-bg27v"]
Jul 30 21:11:27.260032 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.260013    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"]
Jul 30 21:11:27.263414 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:27.263395    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d59412-797f4dc8cb-fz5q6"]
Jul 30 21:11:28.073147 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:28.073114    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0abf8eab-9386-4490-887b-2bb656be9dd9" path="/var/lib/kubelet/pods/0abf8eab-9386-4490-887b-2bb656be9dd9/volumes"
Jul 30 21:11:28.073524 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:28.073485    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2" path="/var/lib/kubelet/pods/cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2/volumes"
Jul 30 21:11:28.073838 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:28.073824    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df87db58-e01c-4e5f-8b1f-695db4352ab6" path="/var/lib/kubelet/pods/df87db58-e01c-4e5f-8b1f-695db4352ab6/volumes"
Jul 30 21:11:44.010631 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.010536    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"]
Jul 30 21:11:44.011128 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.010867    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv" podUID="76e737cf-579b-4ac5-8a85-ca597f7d3b21" containerName="maas-api" containerID="cri-o://2b5cce8baa4abf67dbdb5815a0f9db47870b1c867a21918f7822f593ef805c76" gracePeriod=30
Jul 30 21:11:44.060673 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.060637    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"]
Jul 30 21:11:44.060907 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.060881    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb" podUID="bc8f1782-30d3-44de-a39d-0771b86dbe42" containerName="payload-processing" containerID="cri-o://945ccf25ae4b982bc324279e143bc705288cd1b988e10ca6a7e5a5c346e90b45" gracePeriod=30
Jul 30 21:11:44.076020 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.075978    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"]
Jul 30 21:11:44.076229 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.076193    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72" podUID="75d7a5c7-3bc7-4855-95fe-5057c6bf79ff" containerName="payload-pre-processing" containerID="cri-o://551b2b21b0d8dc2a00227523e9f17ce829378fac2694da04ae618f2b39b2d998" gracePeriod=30
Jul 30 21:11:44.294929 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.294893    2547 generic.go:356] "Generic (PLEG): container finished" podID="75d7a5c7-3bc7-4855-95fe-5057c6bf79ff" containerID="551b2b21b0d8dc2a00227523e9f17ce829378fac2694da04ae618f2b39b2d998" exitCode=0
Jul 30 21:11:44.295132 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.295070    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72" event={"ID":"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff","Type":"ContainerDied","Data":"551b2b21b0d8dc2a00227523e9f17ce829378fac2694da04ae618f2b39b2d998"}
Jul 30 21:11:44.296796 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.296766    2547 generic.go:356] "Generic (PLEG): container finished" podID="bc8f1782-30d3-44de-a39d-0771b86dbe42" containerID="945ccf25ae4b982bc324279e143bc705288cd1b988e10ca6a7e5a5c346e90b45" exitCode=0
Jul 30 21:11:44.296913 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.296850    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb" event={"ID":"bc8f1782-30d3-44de-a39d-0771b86dbe42","Type":"ContainerDied","Data":"945ccf25ae4b982bc324279e143bc705288cd1b988e10ca6a7e5a5c346e90b45"}
Jul 30 21:11:44.298204 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.298180    2547 generic.go:356] "Generic (PLEG): container finished" podID="76e737cf-579b-4ac5-8a85-ca597f7d3b21" containerID="2b5cce8baa4abf67dbdb5815a0f9db47870b1c867a21918f7822f593ef805c76" exitCode=0
Jul 30 21:11:44.298298 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.298234    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv" event={"ID":"76e737cf-579b-4ac5-8a85-ca597f7d3b21","Type":"ContainerDied","Data":"2b5cce8baa4abf67dbdb5815a0f9db47870b1c867a21918f7822f593ef805c76"}
Jul 30 21:11:44.359775 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.359751    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:11:44.407082 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.407058    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:11:44.409209 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.409188    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:11:44.463961 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.463932    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kszwq\" (UniqueName: \"kubernetes.io/projected/bc8f1782-30d3-44de-a39d-0771b86dbe42-kube-api-access-kszwq\") pod \"bc8f1782-30d3-44de-a39d-0771b86dbe42\" (UID: \"bc8f1782-30d3-44de-a39d-0771b86dbe42\") "
Jul 30 21:11:44.464093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.464000    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc8f1782-30d3-44de-a39d-0771b86dbe42-plugins-config-volume\") pod \"bc8f1782-30d3-44de-a39d-0771b86dbe42\" (UID: \"bc8f1782-30d3-44de-a39d-0771b86dbe42\") "
Jul 30 21:11:44.464093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.464057    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4rkp\" (UniqueName: \"kubernetes.io/projected/76e737cf-579b-4ac5-8a85-ca597f7d3b21-kube-api-access-g4rkp\") pod \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") "
Jul 30 21:11:44.464205 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.464122    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6474b\" (UniqueName: \"kubernetes.io/projected/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-kube-api-access-6474b\") pod \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\" (UID: \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\") "
Jul 30 21:11:44.464205 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.464174    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76e737cf-579b-4ac5-8a85-ca597f7d3b21-openshift-service-ca\") pod \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") "
Jul 30 21:11:44.464322 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.464215    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76e737cf-579b-4ac5-8a85-ca597f7d3b21-maas-api-tls\") pod \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\" (UID: \"76e737cf-579b-4ac5-8a85-ca597f7d3b21\") "
Jul 30 21:11:44.464322 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.464276    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-plugins-config-volume\") pod \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\" (UID: \"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff\") "
Jul 30 21:11:44.464438 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.464371    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc8f1782-30d3-44de-a39d-0771b86dbe42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc8f1782-30d3-44de-a39d-0771b86dbe42" (UID: "bc8f1782-30d3-44de-a39d-0771b86dbe42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:11:44.464922 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.464741    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc8f1782-30d3-44de-a39d-0771b86dbe42-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:44.464922 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.464747    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "75d7a5c7-3bc7-4855-95fe-5057c6bf79ff" (UID: "75d7a5c7-3bc7-4855-95fe-5057c6bf79ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:11:44.465090 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.465050    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76e737cf-579b-4ac5-8a85-ca597f7d3b21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "76e737cf-579b-4ac5-8a85-ca597f7d3b21" (UID: "76e737cf-579b-4ac5-8a85-ca597f7d3b21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:11:44.466399 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.466362    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-kube-api-access-6474b" (OuterVolumeSpecName: "kube-api-access-6474b") pod "75d7a5c7-3bc7-4855-95fe-5057c6bf79ff" (UID: "75d7a5c7-3bc7-4855-95fe-5057c6bf79ff"). InnerVolumeSpecName "kube-api-access-6474b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:11:44.466488 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.466418    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76e737cf-579b-4ac5-8a85-ca597f7d3b21-kube-api-access-g4rkp" (OuterVolumeSpecName: "kube-api-access-g4rkp") pod "76e737cf-579b-4ac5-8a85-ca597f7d3b21" (UID: "76e737cf-579b-4ac5-8a85-ca597f7d3b21"). InnerVolumeSpecName "kube-api-access-g4rkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:11:44.466776 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.466757    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc8f1782-30d3-44de-a39d-0771b86dbe42-kube-api-access-kszwq" (OuterVolumeSpecName: "kube-api-access-kszwq") pod "bc8f1782-30d3-44de-a39d-0771b86dbe42" (UID: "bc8f1782-30d3-44de-a39d-0771b86dbe42"). InnerVolumeSpecName "kube-api-access-kszwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:11:44.467042 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.467024    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76e737cf-579b-4ac5-8a85-ca597f7d3b21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "76e737cf-579b-4ac5-8a85-ca597f7d3b21" (UID: "76e737cf-579b-4ac5-8a85-ca597f7d3b21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:11:44.565448 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.565391    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4rkp\" (UniqueName: \"kubernetes.io/projected/76e737cf-579b-4ac5-8a85-ca597f7d3b21-kube-api-access-g4rkp\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:44.565448 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.565415    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6474b\" (UniqueName: \"kubernetes.io/projected/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-kube-api-access-6474b\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:44.565448 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.565424    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76e737cf-579b-4ac5-8a85-ca597f7d3b21-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:44.565448 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.565434    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76e737cf-579b-4ac5-8a85-ca597f7d3b21-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:44.565448 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.565443    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:44.565448 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:44.565451    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kszwq\" (UniqueName: \"kubernetes.io/projected/bc8f1782-30d3-44de-a39d-0771b86dbe42-kube-api-access-kszwq\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:11:45.306222 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.306150    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"
Jul 30 21:11:45.306222 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.306200    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72" event={"ID":"75d7a5c7-3bc7-4855-95fe-5057c6bf79ff","Type":"ContainerDied","Data":"957bc6fb6777ea47f349067415a706234e86fda41c0b5aa0101f90dac42acf17"}
Jul 30 21:11:45.306737 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.306272    2547 scope.go:117] "RemoveContainer" containerID="551b2b21b0d8dc2a00227523e9f17ce829378fac2694da04ae618f2b39b2d998"
Jul 30 21:11:45.307825 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.307790    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb" event={"ID":"bc8f1782-30d3-44de-a39d-0771b86dbe42","Type":"ContainerDied","Data":"28bdeb6cddc569da54ddf3a643fba6b85ca1928e5e7f6a168338587141c53b35"}
Jul 30 21:11:45.307825 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.307807    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"
Jul 30 21:11:45.309099 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.309078    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv" event={"ID":"76e737cf-579b-4ac5-8a85-ca597f7d3b21","Type":"ContainerDied","Data":"90ab77c9b008d5eff03e61673e49fba535a4b569649ff4f14d4a85cf5e114682"}
Jul 30 21:11:45.309201 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.309123    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"
Jul 30 21:11:45.315650 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.315631    2547 scope.go:117] "RemoveContainer" containerID="945ccf25ae4b982bc324279e143bc705288cd1b988e10ca6a7e5a5c346e90b45"
Jul 30 21:11:45.324539 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.324522    2547 scope.go:117] "RemoveContainer" containerID="2b5cce8baa4abf67dbdb5815a0f9db47870b1c867a21918f7822f593ef805c76"
Jul 30 21:11:45.333989 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.333967    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"]
Jul 30 21:11:45.337355 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.337335    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3230ac-597c8bbc5b-zgt72"]
Jul 30 21:11:45.350259 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.350232    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"]
Jul 30 21:11:45.353763 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.353744    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3230ac-5f69456d54-4bngv"]
Jul 30 21:11:45.364332 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.364312    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"]
Jul 30 21:11:45.367330 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:45.367313    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3230ac-7d6f66c6b8-k4qkb"]
Jul 30 21:11:46.072648 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:46.072620    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75d7a5c7-3bc7-4855-95fe-5057c6bf79ff" path="/var/lib/kubelet/pods/75d7a5c7-3bc7-4855-95fe-5057c6bf79ff/volumes"
Jul 30 21:11:46.073011 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:46.072996    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76e737cf-579b-4ac5-8a85-ca597f7d3b21" path="/var/lib/kubelet/pods/76e737cf-579b-4ac5-8a85-ca597f7d3b21/volumes"
Jul 30 21:11:46.073306 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:46.073293    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc8f1782-30d3-44de-a39d-0771b86dbe42" path="/var/lib/kubelet/pods/bc8f1782-30d3-44de-a39d-0771b86dbe42/volumes"
Jul 30 21:11:56.105030 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.104997    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"]
Jul 30 21:11:56.105529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105442    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8f1782-30d3-44de-a39d-0771b86dbe42" containerName="payload-processing"
Jul 30 21:11:56.105529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105457    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8f1782-30d3-44de-a39d-0771b86dbe42" containerName="payload-processing"
Jul 30 21:11:56.105529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105468    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e737cf-579b-4ac5-8a85-ca597f7d3b21" containerName="maas-api"
Jul 30 21:11:56.105529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105476    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e737cf-579b-4ac5-8a85-ca597f7d3b21" containerName="maas-api"
Jul 30 21:11:56.105529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105493    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75d7a5c7-3bc7-4855-95fe-5057c6bf79ff" containerName="payload-pre-processing"
Jul 30 21:11:56.105529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105502    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="75d7a5c7-3bc7-4855-95fe-5057c6bf79ff" containerName="payload-pre-processing"
Jul 30 21:11:56.105529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105518    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df87db58-e01c-4e5f-8b1f-695db4352ab6" containerName="payload-pre-processing"
Jul 30 21:11:56.105529 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105527    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="df87db58-e01c-4e5f-8b1f-695db4352ab6" containerName="payload-pre-processing"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105545    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2" containerName="maas-api"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105553    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2" containerName="maas-api"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105565    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0abf8eab-9386-4490-887b-2bb656be9dd9" containerName="payload-processing"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105570    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="0abf8eab-9386-4490-887b-2bb656be9dd9" containerName="payload-processing"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105625    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="0abf8eab-9386-4490-887b-2bb656be9dd9" containerName="payload-processing"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105633    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc8f1782-30d3-44de-a39d-0771b86dbe42" containerName="payload-processing"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105639    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="76e737cf-579b-4ac5-8a85-ca597f7d3b21" containerName="maas-api"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105647    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="cecbf90b-1d6a-4a16-9d0d-10767b6ae4e2" containerName="maas-api"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105653    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="df87db58-e01c-4e5f-8b1f-695db4352ab6" containerName="payload-pre-processing"
Jul 30 21:11:56.105884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.105657    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="75d7a5c7-3bc7-4855-95fe-5057c6bf79ff" containerName="payload-pre-processing"
Jul 30 21:11:56.110638 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.110617    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.113711 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.113684    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c6145d-serving-cert\""
Jul 30 21:11:56.142972 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.142950    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"]
Jul 30 21:11:56.264289 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.264252    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4kk2\" (UniqueName: \"kubernetes.io/projected/db0faaca-1a86-42e3-b548-e973f5c24650-kube-api-access-t4kk2\") pod \"maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.264459 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.264431    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0faaca-1a86-42e3-b548-e973f5c24650-openshift-service-ca\") pod \"maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.264552 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.264526    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db0faaca-1a86-42e3-b548-e973f5c24650-maas-api-tls\") pod \"maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.365422 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.365342    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4kk2\" (UniqueName: \"kubernetes.io/projected/db0faaca-1a86-42e3-b548-e973f5c24650-kube-api-access-t4kk2\") pod \"maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.365422 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.365390    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0faaca-1a86-42e3-b548-e973f5c24650-openshift-service-ca\") pod \"maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.365628 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.365477    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db0faaca-1a86-42e3-b548-e973f5c24650-maas-api-tls\") pod \"maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.366136 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.366106    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0faaca-1a86-42e3-b548-e973f5c24650-openshift-service-ca\") pod \"maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.368423 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.368401    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db0faaca-1a86-42e3-b548-e973f5c24650-maas-api-tls\") pod \"maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.378883 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.378855    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4kk2\" (UniqueName: \"kubernetes.io/projected/db0faaca-1a86-42e3-b548-e973f5c24650-kube-api-access-t4kk2\") pod \"maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.420967 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.420942    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:56.563983 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.563957    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"]
Jul 30 21:11:56.565806 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:11:56.565768    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb0faaca_1a86_42e3_b548_e973f5c24650.slice/crio-7345ff7af35927a1f179409109d58e0bf2a26548dedaf426a8e48d97c4b9a7d8 WatchSource:0}: Error finding container 7345ff7af35927a1f179409109d58e0bf2a26548dedaf426a8e48d97c4b9a7d8: Status 404 returned error can't find the container with id 7345ff7af35927a1f179409109d58e0bf2a26548dedaf426a8e48d97c4b9a7d8
Jul 30 21:11:56.567092 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.567075    2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 21:11:56.741922 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.741847    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"]
Jul 30 21:11:56.746581 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.746560    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:11:56.753593 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.753571    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-c6145d-dockercfg-m6s5h\""
Jul 30 21:11:56.754372 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.754350    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-c6145d\""
Jul 30 21:11:56.761164 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.761142    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"]
Jul 30 21:11:56.770046 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.770023    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"]
Jul 30 21:11:56.773466 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.773450    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:11:56.786415 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.786393    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"]
Jul 30 21:11:56.870588 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.870552    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjjx9\" (UniqueName: \"kubernetes.io/projected/056aa3b8-855b-40fb-bbe8-acb263260449-kube-api-access-kjjx9\") pod \"payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l\" (UID: \"056aa3b8-855b-40fb-bbe8-acb263260449\") " pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:11:56.870773 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.870614    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/056aa3b8-855b-40fb-bbe8-acb263260449-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l\" (UID: \"056aa3b8-855b-40fb-bbe8-acb263260449\") " pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:11:56.870773 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.870657    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad5418a-1967-454b-a852-62ab35e899bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9\" (UID: \"7ad5418a-1967-454b-a852-62ab35e899bc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:11:56.870773 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.870675    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdgv6\" (UniqueName: \"kubernetes.io/projected/7ad5418a-1967-454b-a852-62ab35e899bc-kube-api-access-vdgv6\") pod \"payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9\" (UID: \"7ad5418a-1967-454b-a852-62ab35e899bc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:11:56.972122 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.972091    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjjx9\" (UniqueName: \"kubernetes.io/projected/056aa3b8-855b-40fb-bbe8-acb263260449-kube-api-access-kjjx9\") pod \"payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l\" (UID: \"056aa3b8-855b-40fb-bbe8-acb263260449\") " pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:11:56.972257 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.972135    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/056aa3b8-855b-40fb-bbe8-acb263260449-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l\" (UID: \"056aa3b8-855b-40fb-bbe8-acb263260449\") " pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:11:56.972257 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.972162    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad5418a-1967-454b-a852-62ab35e899bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9\" (UID: \"7ad5418a-1967-454b-a852-62ab35e899bc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:11:56.972257 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.972181    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdgv6\" (UniqueName: \"kubernetes.io/projected/7ad5418a-1967-454b-a852-62ab35e899bc-kube-api-access-vdgv6\") pod \"payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9\" (UID: \"7ad5418a-1967-454b-a852-62ab35e899bc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:11:56.972894 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.972876    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad5418a-1967-454b-a852-62ab35e899bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9\" (UID: \"7ad5418a-1967-454b-a852-62ab35e899bc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:11:56.972948 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.972883    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/056aa3b8-855b-40fb-bbe8-acb263260449-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l\" (UID: \"056aa3b8-855b-40fb-bbe8-acb263260449\") " pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:11:56.979916 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.979894    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdgv6\" (UniqueName: \"kubernetes.io/projected/7ad5418a-1967-454b-a852-62ab35e899bc-kube-api-access-vdgv6\") pod \"payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9\" (UID: \"7ad5418a-1967-454b-a852-62ab35e899bc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:11:56.979916 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:56.979907    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjjx9\" (UniqueName: \"kubernetes.io/projected/056aa3b8-855b-40fb-bbe8-acb263260449-kube-api-access-kjjx9\") pod \"payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l\" (UID: \"056aa3b8-855b-40fb-bbe8-acb263260449\") " pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:11:57.056926 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.056876    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:11:57.086573 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.086553    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:11:57.231921 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.231894    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"]
Jul 30 21:11:57.233672 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:11:57.233647    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad5418a_1967_454b_a852_62ab35e899bc.slice/crio-c07f5ad2c9a9e5461348637edc660181490f0667f32d4428105f153fbb441d96 WatchSource:0}: Error finding container c07f5ad2c9a9e5461348637edc660181490f0667f32d4428105f153fbb441d96: Status 404 returned error can't find the container with id c07f5ad2c9a9e5461348637edc660181490f0667f32d4428105f153fbb441d96
Jul 30 21:11:57.375389 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.375353    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx" event={"ID":"db0faaca-1a86-42e3-b548-e973f5c24650","Type":"ContainerStarted","Data":"b21feefbae4ad68b9e4db60f11056b7407a7fc83a985ebd879ac9b56a2ba69d3"}
Jul 30 21:11:57.375536 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.375423    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:11:57.375536 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.375448    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx" event={"ID":"db0faaca-1a86-42e3-b548-e973f5c24650","Type":"ContainerStarted","Data":"7345ff7af35927a1f179409109d58e0bf2a26548dedaf426a8e48d97c4b9a7d8"}
Jul 30 21:11:57.377044 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.377021    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9" event={"ID":"7ad5418a-1967-454b-a852-62ab35e899bc","Type":"ContainerStarted","Data":"c5381aedc50cb3dd4c13c2e60b0264be280bf47200f81138ae1ab931fa13292c"}
Jul 30 21:11:57.377141 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.377066    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9" event={"ID":"7ad5418a-1967-454b-a852-62ab35e899bc","Type":"ContainerStarted","Data":"c07f5ad2c9a9e5461348637edc660181490f0667f32d4428105f153fbb441d96"}
Jul 30 21:11:57.377141 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.377116    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:11:57.415687 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.415638    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx" podStartSLOduration=0.905002934 podStartE2EDuration="1.415623277s" totalImagesPullingTime="510.620343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:11:57.397839308 +0000 UTC m=+2795.858798122" watchObservedRunningTime="2026-07-30 21:11:57.415623277 +0000 UTC m=+2795.876582089"
Jul 30 21:11:57.415868 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.415847    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9" podStartSLOduration=1.415841046 podStartE2EDuration="1.415841046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:11:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:11:57.413686415 +0000 UTC m=+2795.874645228" watchObservedRunningTime="2026-07-30 21:11:57.415841046 +0000 UTC m=+2795.876799858"
Jul 30 21:11:57.461261 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:57.461234    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"]
Jul 30 21:11:57.464901 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:11:57.464870    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056aa3b8_855b_40fb_bbe8_acb263260449.slice/crio-edeb4d84595b581191d3360fad98bafc71324889fd5e5bebc35867128d2051a5 WatchSource:0}: Error finding container edeb4d84595b581191d3360fad98bafc71324889fd5e5bebc35867128d2051a5: Status 404 returned error can't find the container with id edeb4d84595b581191d3360fad98bafc71324889fd5e5bebc35867128d2051a5
Jul 30 21:11:58.383715 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:58.383662    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l" event={"ID":"056aa3b8-855b-40fb-bbe8-acb263260449","Type":"ContainerStarted","Data":"29fd26556d52588c185b65bf29510e027fda0e92c2785e8253f71fa8df4fa198"}
Jul 30 21:11:58.384177 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:58.383733    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l" event={"ID":"056aa3b8-855b-40fb-bbe8-acb263260449","Type":"ContainerStarted","Data":"edeb4d84595b581191d3360fad98bafc71324889fd5e5bebc35867128d2051a5"}
Jul 30 21:11:58.384177 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:58.384076    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:11:58.400104 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:11:58.400061    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l" podStartSLOduration=2.400048428 podStartE2EDuration="2.400048428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:11:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:11:58.398087774 +0000 UTC m=+2796.859046586" watchObservedRunningTime="2026-07-30 21:11:58.400048428 +0000 UTC m=+2796.861007239"
Jul 30 21:12:03.389693 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:03.389662    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:12:08.385802 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:08.385763    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:12:10.396906 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:10.396870    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:12:25.234469 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.234437    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"]
Jul 30 21:12:25.238711 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.238685    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.241272 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.241251    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-039a87-serving-cert\""
Jul 30 21:12:25.252115 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.252019    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"]
Jul 30 21:12:25.302535 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.302509    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dae5e32b-bfa5-46f8-808d-990e67071836-maas-api-tls\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.302678 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.302546    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dae5e32b-bfa5-46f8-808d-990e67071836-openshift-service-ca\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.302678 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.302576    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkgxz\" (UniqueName: \"kubernetes.io/projected/dae5e32b-bfa5-46f8-808d-990e67071836-kube-api-access-kkgxz\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.403621 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.403588    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkgxz\" (UniqueName: \"kubernetes.io/projected/dae5e32b-bfa5-46f8-808d-990e67071836-kube-api-access-kkgxz\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.403802 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.403731    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dae5e32b-bfa5-46f8-808d-990e67071836-maas-api-tls\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.403802 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.403776    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dae5e32b-bfa5-46f8-808d-990e67071836-openshift-service-ca\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.403936 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:12:25.403896    2547 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-serving-cert: secret "maas-api-e2e-api-b-039a87-serving-cert" not found
Jul 30 21:12:25.403998 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:12:25.403984    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae5e32b-bfa5-46f8-808d-990e67071836-maas-api-tls podName:dae5e32b-bfa5-46f8-808d-990e67071836 nodeName:}" failed. No retries permitted until 2026-07-30 21:12:25.903962302 +0000 UTC m=+2824.364921106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dae5e32b-bfa5-46f8-808d-990e67071836-maas-api-tls") pod "maas-api-e2e-api-b-039a87-6df9759647-5n9ts" (UID: "dae5e32b-bfa5-46f8-808d-990e67071836") : secret "maas-api-e2e-api-b-039a87-serving-cert" not found
Jul 30 21:12:25.404401 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.404379    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dae5e32b-bfa5-46f8-808d-990e67071836-openshift-service-ca\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.413303 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.413282    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkgxz\" (UniqueName: \"kubernetes.io/projected/dae5e32b-bfa5-46f8-808d-990e67071836-kube-api-access-kkgxz\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.857971 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.857937    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"]
Jul 30 21:12:25.861757 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.861734    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:25.864725 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.864687    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-039a87\""
Jul 30 21:12:25.871018 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.870995    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"]
Jul 30 21:12:25.890077 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.890049    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"]
Jul 30 21:12:25.893613 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.893592    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:12:25.896151 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.896131    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-039a87-dockercfg-cdnv4\""
Jul 30 21:12:25.908283 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.908260    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"]
Jul 30 21:12:25.908433 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.908409    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v\" (UID: \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:25.908535 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.908522    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s27xz\" (UniqueName: \"kubernetes.io/projected/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-kube-api-access-s27xz\") pod \"payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v\" (UID: \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:25.908576 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.908553    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5fd67c-4d81-48c5-9741-6852192662e7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-039a87-66bd668946-zkllk\" (UID: \"ee5fd67c-4d81-48c5-9741-6852192662e7\") " pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:12:25.908616 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.908592    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dae5e32b-bfa5-46f8-808d-990e67071836-maas-api-tls\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:25.908737 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.908715    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77dlt\" (UniqueName: \"kubernetes.io/projected/ee5fd67c-4d81-48c5-9741-6852192662e7-kube-api-access-77dlt\") pod \"payload-processing-e2e-api-b-039a87-66bd668946-zkllk\" (UID: \"ee5fd67c-4d81-48c5-9741-6852192662e7\") " pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:12:25.911353 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:25.911327    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dae5e32b-bfa5-46f8-808d-990e67071836-maas-api-tls\") pod \"maas-api-e2e-api-b-039a87-6df9759647-5n9ts\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:26.009519 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.009486    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v\" (UID: \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:26.009677 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.009562    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s27xz\" (UniqueName: \"kubernetes.io/projected/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-kube-api-access-s27xz\") pod \"payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v\" (UID: \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:26.009677 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.009590    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5fd67c-4d81-48c5-9741-6852192662e7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-039a87-66bd668946-zkllk\" (UID: \"ee5fd67c-4d81-48c5-9741-6852192662e7\") " pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:12:26.009677 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.009649    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77dlt\" (UniqueName: \"kubernetes.io/projected/ee5fd67c-4d81-48c5-9741-6852192662e7-kube-api-access-77dlt\") pod \"payload-processing-e2e-api-b-039a87-66bd668946-zkllk\" (UID: \"ee5fd67c-4d81-48c5-9741-6852192662e7\") " pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:12:26.010351 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.010322    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v\" (UID: \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:26.010446 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.010427    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5fd67c-4d81-48c5-9741-6852192662e7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-039a87-66bd668946-zkllk\" (UID: \"ee5fd67c-4d81-48c5-9741-6852192662e7\") " pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:12:26.017656 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.017637    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77dlt\" (UniqueName: \"kubernetes.io/projected/ee5fd67c-4d81-48c5-9741-6852192662e7-kube-api-access-77dlt\") pod \"payload-processing-e2e-api-b-039a87-66bd668946-zkllk\" (UID: \"ee5fd67c-4d81-48c5-9741-6852192662e7\") " pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:12:26.021440 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.021416    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s27xz\" (UniqueName: \"kubernetes.io/projected/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-kube-api-access-s27xz\") pod \"payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v\" (UID: \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:26.154611 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.154528    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:26.174302 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.174280    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:26.205995 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.205968    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:12:26.323181 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.323149    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"]
Jul 30 21:12:26.326142 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:12:26.325961    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddae5e32b_bfa5_46f8_808d_990e67071836.slice/crio-7d16299de52f16116b0b5000edf1dbec90d17446af4139f16e9faeafdc323529 WatchSource:0}: Error finding container 7d16299de52f16116b0b5000edf1dbec90d17446af4139f16e9faeafdc323529: Status 404 returned error can't find the container with id 7d16299de52f16116b0b5000edf1dbec90d17446af4139f16e9faeafdc323529
Jul 30 21:12:26.357217 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.357187    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"]
Jul 30 21:12:26.357791 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:12:26.357758    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0fb33e_1601_490e_bcd8_b7fa24ecaab5.slice/crio-3a1b485d00edf53fc456cd11ca1524065eb63e9bb6ed9b1ffb3d3da216436732 WatchSource:0}: Error finding container 3a1b485d00edf53fc456cd11ca1524065eb63e9bb6ed9b1ffb3d3da216436732: Status 404 returned error can't find the container with id 3a1b485d00edf53fc456cd11ca1524065eb63e9bb6ed9b1ffb3d3da216436732
Jul 30 21:12:26.386255 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.386231    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"]
Jul 30 21:12:26.394872 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:12:26.394846    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5fd67c_4d81_48c5_9741_6852192662e7.slice/crio-1d98c83a679d40e063882757e3da2219020552e2dd449bee414af2892aaa1cd8 WatchSource:0}: Error finding container 1d98c83a679d40e063882757e3da2219020552e2dd449bee414af2892aaa1cd8: Status 404 returned error can't find the container with id 1d98c83a679d40e063882757e3da2219020552e2dd449bee414af2892aaa1cd8
Jul 30 21:12:26.541774 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.541744    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk" event={"ID":"ee5fd67c-4d81-48c5-9741-6852192662e7","Type":"ContainerStarted","Data":"e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2"}
Jul 30 21:12:26.541944 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.541804    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:12:26.541944 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.541825    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk" event={"ID":"ee5fd67c-4d81-48c5-9741-6852192662e7","Type":"ContainerStarted","Data":"1d98c83a679d40e063882757e3da2219020552e2dd449bee414af2892aaa1cd8"}
Jul 30 21:12:26.543310 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.543286    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v" event={"ID":"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5","Type":"ContainerStarted","Data":"687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb"}
Jul 30 21:12:26.543425 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.543329    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:26.543425 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.543345    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v" event={"ID":"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5","Type":"ContainerStarted","Data":"3a1b485d00edf53fc456cd11ca1524065eb63e9bb6ed9b1ffb3d3da216436732"}
Jul 30 21:12:26.544468 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.544440    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts" event={"ID":"dae5e32b-bfa5-46f8-808d-990e67071836","Type":"ContainerStarted","Data":"7d16299de52f16116b0b5000edf1dbec90d17446af4139f16e9faeafdc323529"}
Jul 30 21:12:26.559540 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.559491    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk" podStartSLOduration=1.559474936 podStartE2EDuration="1.559474936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:12:26.557143169 +0000 UTC m=+2825.018101980" watchObservedRunningTime="2026-07-30 21:12:26.559474936 +0000 UTC m=+2825.020433727"
Jul 30 21:12:26.572889 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:26.572847    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v" podStartSLOduration=1.572834702 podStartE2EDuration="1.572834702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:12:26.570965789 +0000 UTC m=+2825.031924604" watchObservedRunningTime="2026-07-30 21:12:26.572834702 +0000 UTC m=+2825.033793514"
Jul 30 21:12:27.552880 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:27.552838    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts" event={"ID":"dae5e32b-bfa5-46f8-808d-990e67071836","Type":"ContainerStarted","Data":"526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35"}
Jul 30 21:12:27.553378 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:27.553122    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:27.569692 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:27.569640    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts" podStartSLOduration=2.243983596 podStartE2EDuration="2.569622614s" totalImagesPullingTime="325.639018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:12:27.568614669 +0000 UTC m=+2826.029573481" watchObservedRunningTime="2026-07-30 21:12:27.569622614 +0000 UTC m=+2826.030581429"
Jul 30 21:12:33.562869 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:33.562839    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:12:37.555931 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:37.555894    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:12:37.556377 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:12:37.555977    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:13:07.576017 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.575979    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"]
Jul 30 21:13:07.576585 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.576277    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx" podUID="db0faaca-1a86-42e3-b548-e973f5c24650" containerName="maas-api" containerID="cri-o://b21feefbae4ad68b9e4db60f11056b7407a7fc83a985ebd879ac9b56a2ba69d3" gracePeriod=30
Jul 30 21:13:07.637439 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.637403    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"]
Jul 30 21:13:07.637861 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.637662    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l" podUID="056aa3b8-855b-40fb-bbe8-acb263260449" containerName="payload-processing" containerID="cri-o://29fd26556d52588c185b65bf29510e027fda0e92c2785e8253f71fa8df4fa198" gracePeriod=30
Jul 30 21:13:07.640688 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.640663    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"]
Jul 30 21:13:07.641068 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.640886    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9" podUID="7ad5418a-1967-454b-a852-62ab35e899bc" containerName="payload-pre-processing" containerID="cri-o://c5381aedc50cb3dd4c13c2e60b0264be280bf47200f81138ae1ab931fa13292c" gracePeriod=30
Jul 30 21:13:07.789984 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.789917    2547 generic.go:356] "Generic (PLEG): container finished" podID="db0faaca-1a86-42e3-b548-e973f5c24650" containerID="b21feefbae4ad68b9e4db60f11056b7407a7fc83a985ebd879ac9b56a2ba69d3" exitCode=0
Jul 30 21:13:07.790137 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.790008    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx" event={"ID":"db0faaca-1a86-42e3-b548-e973f5c24650","Type":"ContainerDied","Data":"b21feefbae4ad68b9e4db60f11056b7407a7fc83a985ebd879ac9b56a2ba69d3"}
Jul 30 21:13:07.791883 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.791862    2547 generic.go:356] "Generic (PLEG): container finished" podID="056aa3b8-855b-40fb-bbe8-acb263260449" containerID="29fd26556d52588c185b65bf29510e027fda0e92c2785e8253f71fa8df4fa198" exitCode=0
Jul 30 21:13:07.791960 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.791932    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l" event={"ID":"056aa3b8-855b-40fb-bbe8-acb263260449","Type":"ContainerDied","Data":"29fd26556d52588c185b65bf29510e027fda0e92c2785e8253f71fa8df4fa198"}
Jul 30 21:13:07.793796 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.793772    2547 generic.go:356] "Generic (PLEG): container finished" podID="7ad5418a-1967-454b-a852-62ab35e899bc" containerID="c5381aedc50cb3dd4c13c2e60b0264be280bf47200f81138ae1ab931fa13292c" exitCode=0
Jul 30 21:13:07.793851 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.793840    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9" event={"ID":"7ad5418a-1967-454b-a852-62ab35e899bc","Type":"ContainerDied","Data":"c5381aedc50cb3dd4c13c2e60b0264be280bf47200f81138ae1ab931fa13292c"}
Jul 30 21:13:07.908091 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.908061    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:13:07.949897 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.949871    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:13:07.957684 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:07.957662    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:13:08.059416 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059391    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad5418a-1967-454b-a852-62ab35e899bc-plugins-config-volume\") pod \"7ad5418a-1967-454b-a852-62ab35e899bc\" (UID: \"7ad5418a-1967-454b-a852-62ab35e899bc\") "
Jul 30 21:13:08.059580 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059428    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db0faaca-1a86-42e3-b548-e973f5c24650-maas-api-tls\") pod \"db0faaca-1a86-42e3-b548-e973f5c24650\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") "
Jul 30 21:13:08.059580 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059487    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4kk2\" (UniqueName: \"kubernetes.io/projected/db0faaca-1a86-42e3-b548-e973f5c24650-kube-api-access-t4kk2\") pod \"db0faaca-1a86-42e3-b548-e973f5c24650\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") "
Jul 30 21:13:08.059580 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059512    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/056aa3b8-855b-40fb-bbe8-acb263260449-plugins-config-volume\") pod \"056aa3b8-855b-40fb-bbe8-acb263260449\" (UID: \"056aa3b8-855b-40fb-bbe8-acb263260449\") "
Jul 30 21:13:08.059580 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059543    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0faaca-1a86-42e3-b548-e973f5c24650-openshift-service-ca\") pod \"db0faaca-1a86-42e3-b548-e973f5c24650\" (UID: \"db0faaca-1a86-42e3-b548-e973f5c24650\") "
Jul 30 21:13:08.059860 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059617    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjjx9\" (UniqueName: \"kubernetes.io/projected/056aa3b8-855b-40fb-bbe8-acb263260449-kube-api-access-kjjx9\") pod \"056aa3b8-855b-40fb-bbe8-acb263260449\" (UID: \"056aa3b8-855b-40fb-bbe8-acb263260449\") "
Jul 30 21:13:08.059860 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059641    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdgv6\" (UniqueName: \"kubernetes.io/projected/7ad5418a-1967-454b-a852-62ab35e899bc-kube-api-access-vdgv6\") pod \"7ad5418a-1967-454b-a852-62ab35e899bc\" (UID: \"7ad5418a-1967-454b-a852-62ab35e899bc\") "
Jul 30 21:13:08.059984 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059862    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad5418a-1967-454b-a852-62ab35e899bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ad5418a-1967-454b-a852-62ab35e899bc" (UID: "7ad5418a-1967-454b-a852-62ab35e899bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:13:08.059984 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059949    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db0faaca-1a86-42e3-b548-e973f5c24650-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db0faaca-1a86-42e3-b548-e973f5c24650" (UID: "db0faaca-1a86-42e3-b548-e973f5c24650"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:13:08.060098 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.059952    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/056aa3b8-855b-40fb-bbe8-acb263260449-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "056aa3b8-855b-40fb-bbe8-acb263260449" (UID: "056aa3b8-855b-40fb-bbe8-acb263260449"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:13:08.060098 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.060060    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad5418a-1967-454b-a852-62ab35e899bc-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:08.060098 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.060078    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0faaca-1a86-42e3-b548-e973f5c24650-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:08.061957 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.061931    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db0faaca-1a86-42e3-b548-e973f5c24650-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db0faaca-1a86-42e3-b548-e973f5c24650" (UID: "db0faaca-1a86-42e3-b548-e973f5c24650"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:13:08.061957 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.061945    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/056aa3b8-855b-40fb-bbe8-acb263260449-kube-api-access-kjjx9" (OuterVolumeSpecName: "kube-api-access-kjjx9") pod "056aa3b8-855b-40fb-bbe8-acb263260449" (UID: "056aa3b8-855b-40fb-bbe8-acb263260449"). InnerVolumeSpecName "kube-api-access-kjjx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:13:08.061957 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.061936    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db0faaca-1a86-42e3-b548-e973f5c24650-kube-api-access-t4kk2" (OuterVolumeSpecName: "kube-api-access-t4kk2") pod "db0faaca-1a86-42e3-b548-e973f5c24650" (UID: "db0faaca-1a86-42e3-b548-e973f5c24650"). InnerVolumeSpecName "kube-api-access-t4kk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:13:08.062131 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.061951    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad5418a-1967-454b-a852-62ab35e899bc-kube-api-access-vdgv6" (OuterVolumeSpecName: "kube-api-access-vdgv6") pod "7ad5418a-1967-454b-a852-62ab35e899bc" (UID: "7ad5418a-1967-454b-a852-62ab35e899bc"). InnerVolumeSpecName "kube-api-access-vdgv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:13:08.161146 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.161084    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db0faaca-1a86-42e3-b548-e973f5c24650-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:08.161146 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.161108    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4kk2\" (UniqueName: \"kubernetes.io/projected/db0faaca-1a86-42e3-b548-e973f5c24650-kube-api-access-t4kk2\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:08.161146 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.161120    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/056aa3b8-855b-40fb-bbe8-acb263260449-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:08.161146 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.161129    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjjx9\" (UniqueName: \"kubernetes.io/projected/056aa3b8-855b-40fb-bbe8-acb263260449-kube-api-access-kjjx9\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:08.161146 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.161138    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdgv6\" (UniqueName: \"kubernetes.io/projected/7ad5418a-1967-454b-a852-62ab35e899bc-kube-api-access-vdgv6\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:08.800341 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.800307    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx" event={"ID":"db0faaca-1a86-42e3-b548-e973f5c24650","Type":"ContainerDied","Data":"7345ff7af35927a1f179409109d58e0bf2a26548dedaf426a8e48d97c4b9a7d8"}
Jul 30 21:13:08.800739 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.800369    2547 scope.go:117] "RemoveContainer" containerID="b21feefbae4ad68b9e4db60f11056b7407a7fc83a985ebd879ac9b56a2ba69d3"
Jul 30 21:13:08.800739 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.800318    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"
Jul 30 21:13:08.802354 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.801840    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l" event={"ID":"056aa3b8-855b-40fb-bbe8-acb263260449","Type":"ContainerDied","Data":"edeb4d84595b581191d3360fad98bafc71324889fd5e5bebc35867128d2051a5"}
Jul 30 21:13:08.802354 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.801907    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"
Jul 30 21:13:08.803369 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.803351    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"
Jul 30 21:13:08.803479 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.803407    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9" event={"ID":"7ad5418a-1967-454b-a852-62ab35e899bc","Type":"ContainerDied","Data":"c07f5ad2c9a9e5461348637edc660181490f0667f32d4428105f153fbb441d96"}
Jul 30 21:13:08.810453 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.810438    2547 scope.go:117] "RemoveContainer" containerID="29fd26556d52588c185b65bf29510e027fda0e92c2785e8253f71fa8df4fa198"
Jul 30 21:13:08.818178 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.818163    2547 scope.go:117] "RemoveContainer" containerID="c5381aedc50cb3dd4c13c2e60b0264be280bf47200f81138ae1ab931fa13292c"
Jul 30 21:13:08.823645 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.823625    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"]
Jul 30 21:13:08.826670 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.826652    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c6145d-557d95cdc4-zs86l"]
Jul 30 21:13:08.836683 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.836663    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"]
Jul 30 21:13:08.840020 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.840000    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c6145d-c4966bcc5-xxxxx"]
Jul 30 21:13:08.851817 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.851766    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"]
Jul 30 21:13:08.856056 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:08.856039    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6145d-7c8779c8d8-zg4m9"]
Jul 30 21:13:10.072330 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:10.072299    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="056aa3b8-855b-40fb-bbe8-acb263260449" path="/var/lib/kubelet/pods/056aa3b8-855b-40fb-bbe8-acb263260449/volumes"
Jul 30 21:13:10.072687 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:10.072626    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ad5418a-1967-454b-a852-62ab35e899bc" path="/var/lib/kubelet/pods/7ad5418a-1967-454b-a852-62ab35e899bc/volumes"
Jul 30 21:13:10.072984 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:10.072970    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db0faaca-1a86-42e3-b548-e973f5c24650" path="/var/lib/kubelet/pods/db0faaca-1a86-42e3-b548-e973f5c24650/volumes"
Jul 30 21:13:14.510789 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.510682    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"]
Jul 30 21:13:14.511200 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.511182    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0faaca-1a86-42e3-b548-e973f5c24650" containerName="maas-api"
Jul 30 21:13:14.511247 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.511203    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0faaca-1a86-42e3-b548-e973f5c24650" containerName="maas-api"
Jul 30 21:13:14.511247 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.511218    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad5418a-1967-454b-a852-62ab35e899bc" containerName="payload-pre-processing"
Jul 30 21:13:14.511247 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.511224    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad5418a-1967-454b-a852-62ab35e899bc" containerName="payload-pre-processing"
Jul 30 21:13:14.511247 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.511236    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="056aa3b8-855b-40fb-bbe8-acb263260449" containerName="payload-processing"
Jul 30 21:13:14.511247 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.511243    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="056aa3b8-855b-40fb-bbe8-acb263260449" containerName="payload-processing"
Jul 30 21:13:14.511462 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.511311    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="056aa3b8-855b-40fb-bbe8-acb263260449" containerName="payload-processing"
Jul 30 21:13:14.511462 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.511320    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad5418a-1967-454b-a852-62ab35e899bc" containerName="payload-pre-processing"
Jul 30 21:13:14.511462 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.511328    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0faaca-1a86-42e3-b548-e973f5c24650" containerName="maas-api"
Jul 30 21:13:14.515970 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.515950    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"
Jul 30 21:13:14.518933 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.518910    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zm2hs\""
Jul 30 21:13:14.520887 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.520860    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"]
Jul 30 21:13:14.611250 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.611223    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0316d4f7-016f-4862-b247-c901dadfbaa6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-039a87-br65p\" (UID: \"0316d4f7-016f-4862-b247-c901dadfbaa6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"
Jul 30 21:13:14.712271 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.712246    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0316d4f7-016f-4862-b247-c901dadfbaa6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-039a87-br65p\" (UID: \"0316d4f7-016f-4862-b247-c901dadfbaa6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"
Jul 30 21:13:14.712933 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.712914    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0316d4f7-016f-4862-b247-c901dadfbaa6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-039a87-br65p\" (UID: \"0316d4f7-016f-4862-b247-c901dadfbaa6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"
Jul 30 21:13:14.828442 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.828412    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"
Jul 30 21:13:14.997530 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:14.997504    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"]
Jul 30 21:13:15.000154 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:13:15.000124    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0316d4f7_016f_4862_b247_c901dadfbaa6.slice/crio-6cd0a2898c2f7bba75ad502c95edd847cac4ed318eeb61e193b2e8d3b51ab4f3 WatchSource:0}: Error finding container 6cd0a2898c2f7bba75ad502c95edd847cac4ed318eeb61e193b2e8d3b51ab4f3: Status 404 returned error can't find the container with id 6cd0a2898c2f7bba75ad502c95edd847cac4ed318eeb61e193b2e8d3b51ab4f3
Jul 30 21:13:15.844424 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:15.844390    2547 generic.go:356] "Generic (PLEG): container finished" podID="0316d4f7-016f-4862-b247-c901dadfbaa6" containerID="3b819939bf262c69a53b8217fa4e58aa053212e5e9f6faa1f13acb68086eb248" exitCode=0
Jul 30 21:13:15.844783 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:15.844469    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p" event={"ID":"0316d4f7-016f-4862-b247-c901dadfbaa6","Type":"ContainerDied","Data":"3b819939bf262c69a53b8217fa4e58aa053212e5e9f6faa1f13acb68086eb248"}
Jul 30 21:13:15.844783 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:15.844511    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p" event={"ID":"0316d4f7-016f-4862-b247-c901dadfbaa6","Type":"ContainerStarted","Data":"6cd0a2898c2f7bba75ad502c95edd847cac4ed318eeb61e193b2e8d3b51ab4f3"}
Jul 30 21:13:16.986363 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:16.986333    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"
Jul 30 21:13:17.134502 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:17.134430    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0316d4f7-016f-4862-b247-c901dadfbaa6-maas-api-service-ca\") pod \"0316d4f7-016f-4862-b247-c901dadfbaa6\" (UID: \"0316d4f7-016f-4862-b247-c901dadfbaa6\") "
Jul 30 21:13:17.134824 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:17.134800    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0316d4f7-016f-4862-b247-c901dadfbaa6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0316d4f7-016f-4862-b247-c901dadfbaa6" (UID: "0316d4f7-016f-4862-b247-c901dadfbaa6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:13:17.235793 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:17.235771    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0316d4f7-016f-4862-b247-c901dadfbaa6-maas-api-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:17.858146 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:17.858113    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"
Jul 30 21:13:17.858314 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:17.858144    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p" event={"ID":"0316d4f7-016f-4862-b247-c901dadfbaa6","Type":"ContainerDied","Data":"6cd0a2898c2f7bba75ad502c95edd847cac4ed318eeb61e193b2e8d3b51ab4f3"}
Jul 30 21:13:17.858314 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:17.858205    2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6cd0a2898c2f7bba75ad502c95edd847cac4ed318eeb61e193b2e8d3b51ab4f3"
Jul 30 21:13:29.950785 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:29.950694    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"]
Jul 30 21:13:29.951336 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:29.951071    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts" podUID="dae5e32b-bfa5-46f8-808d-990e67071836" containerName="maas-api" containerID="cri-o://526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35" gracePeriod=30
Jul 30 21:13:30.006577 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.006543    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"]
Jul 30 21:13:30.006827 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.006801    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk" podUID="ee5fd67c-4d81-48c5-9741-6852192662e7" containerName="payload-processing" containerID="cri-o://e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2" gracePeriod=30
Jul 30 21:13:30.011259 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.011231    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"]
Jul 30 21:13:30.013985 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.013952    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v" podUID="cc0fb33e-1601-490e-bcd8-b7fa24ecaab5" containerName="payload-pre-processing" containerID="cri-o://687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb" gracePeriod=30
Jul 30 21:13:30.140051 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.139999    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"]
Jul 30 21:13:30.144730 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.144678    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-039a87-br65p"]
Jul 30 21:13:30.277293 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.277263    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:13:30.281713 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.281680    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:13:30.316081 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.316059    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:13:30.330819 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.330797    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-plugins-config-volume\") pod \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\" (UID: \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\") "
Jul 30 21:13:30.330920 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.330862    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s27xz\" (UniqueName: \"kubernetes.io/projected/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-kube-api-access-s27xz\") pod \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\" (UID: \"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5\") "
Jul 30 21:13:30.331060 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331040    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77dlt\" (UniqueName: \"kubernetes.io/projected/ee5fd67c-4d81-48c5-9741-6852192662e7-kube-api-access-77dlt\") pod \"ee5fd67c-4d81-48c5-9741-6852192662e7\" (UID: \"ee5fd67c-4d81-48c5-9741-6852192662e7\") "
Jul 30 21:13:30.331135 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331077    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkgxz\" (UniqueName: \"kubernetes.io/projected/dae5e32b-bfa5-46f8-808d-990e67071836-kube-api-access-kkgxz\") pod \"dae5e32b-bfa5-46f8-808d-990e67071836\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") "
Jul 30 21:13:30.331135 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331111    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dae5e32b-bfa5-46f8-808d-990e67071836-openshift-service-ca\") pod \"dae5e32b-bfa5-46f8-808d-990e67071836\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") "
Jul 30 21:13:30.331254 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331127    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cc0fb33e-1601-490e-bcd8-b7fa24ecaab5" (UID: "cc0fb33e-1601-490e-bcd8-b7fa24ecaab5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:13:30.331254 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331196    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dae5e32b-bfa5-46f8-808d-990e67071836-maas-api-tls\") pod \"dae5e32b-bfa5-46f8-808d-990e67071836\" (UID: \"dae5e32b-bfa5-46f8-808d-990e67071836\") "
Jul 30 21:13:30.331254 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331228    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5fd67c-4d81-48c5-9741-6852192662e7-plugins-config-volume\") pod \"ee5fd67c-4d81-48c5-9741-6852192662e7\" (UID: \"ee5fd67c-4d81-48c5-9741-6852192662e7\") "
Jul 30 21:13:30.331488 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331466    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dae5e32b-bfa5-46f8-808d-990e67071836-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dae5e32b-bfa5-46f8-808d-990e67071836" (UID: "dae5e32b-bfa5-46f8-808d-990e67071836"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:13:30.331751 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331590    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dae5e32b-bfa5-46f8-808d-990e67071836-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:30.331751 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331609    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:30.331880 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.331799    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee5fd67c-4d81-48c5-9741-6852192662e7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee5fd67c-4d81-48c5-9741-6852192662e7" (UID: "ee5fd67c-4d81-48c5-9741-6852192662e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:13:30.333304 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.333277    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5fd67c-4d81-48c5-9741-6852192662e7-kube-api-access-77dlt" (OuterVolumeSpecName: "kube-api-access-77dlt") pod "ee5fd67c-4d81-48c5-9741-6852192662e7" (UID: "ee5fd67c-4d81-48c5-9741-6852192662e7"). InnerVolumeSpecName "kube-api-access-77dlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:13:30.333554 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.333527    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dae5e32b-bfa5-46f8-808d-990e67071836-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dae5e32b-bfa5-46f8-808d-990e67071836" (UID: "dae5e32b-bfa5-46f8-808d-990e67071836"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:13:30.333748 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.333726    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-kube-api-access-s27xz" (OuterVolumeSpecName: "kube-api-access-s27xz") pod "cc0fb33e-1601-490e-bcd8-b7fa24ecaab5" (UID: "cc0fb33e-1601-490e-bcd8-b7fa24ecaab5"). InnerVolumeSpecName "kube-api-access-s27xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:13:30.333942 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.333918    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dae5e32b-bfa5-46f8-808d-990e67071836-kube-api-access-kkgxz" (OuterVolumeSpecName: "kube-api-access-kkgxz") pod "dae5e32b-bfa5-46f8-808d-990e67071836" (UID: "dae5e32b-bfa5-46f8-808d-990e67071836"). InnerVolumeSpecName "kube-api-access-kkgxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:13:30.432747 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.432720    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s27xz\" (UniqueName: \"kubernetes.io/projected/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5-kube-api-access-s27xz\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:30.432747 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.432742    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77dlt\" (UniqueName: \"kubernetes.io/projected/ee5fd67c-4d81-48c5-9741-6852192662e7-kube-api-access-77dlt\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:30.432747 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.432751    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkgxz\" (UniqueName: \"kubernetes.io/projected/dae5e32b-bfa5-46f8-808d-990e67071836-kube-api-access-kkgxz\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:30.432948 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.432761    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dae5e32b-bfa5-46f8-808d-990e67071836-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:30.432948 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.432771    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee5fd67c-4d81-48c5-9741-6852192662e7-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:13:30.936153 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.936121    2547 generic.go:356] "Generic (PLEG): container finished" podID="cc0fb33e-1601-490e-bcd8-b7fa24ecaab5" containerID="687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb" exitCode=0
Jul 30 21:13:30.936324 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.936165    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"
Jul 30 21:13:30.936324 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.936207    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v" event={"ID":"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5","Type":"ContainerDied","Data":"687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb"}
Jul 30 21:13:30.936324 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.936262    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v" event={"ID":"cc0fb33e-1601-490e-bcd8-b7fa24ecaab5","Type":"ContainerDied","Data":"3a1b485d00edf53fc456cd11ca1524065eb63e9bb6ed9b1ffb3d3da216436732"}
Jul 30 21:13:30.936324 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.936294    2547 scope.go:117] "RemoveContainer" containerID="687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb"
Jul 30 21:13:30.937422 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.937400    2547 generic.go:356] "Generic (PLEG): container finished" podID="dae5e32b-bfa5-46f8-808d-990e67071836" containerID="526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35" exitCode=0
Jul 30 21:13:30.937486 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.937437    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"
Jul 30 21:13:30.937542 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.937476    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts" event={"ID":"dae5e32b-bfa5-46f8-808d-990e67071836","Type":"ContainerDied","Data":"526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35"}
Jul 30 21:13:30.937542 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.937534    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts" event={"ID":"dae5e32b-bfa5-46f8-808d-990e67071836","Type":"ContainerDied","Data":"7d16299de52f16116b0b5000edf1dbec90d17446af4139f16e9faeafdc323529"}
Jul 30 21:13:30.939157 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.939129    2547 generic.go:356] "Generic (PLEG): container finished" podID="ee5fd67c-4d81-48c5-9741-6852192662e7" containerID="e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2" exitCode=0
Jul 30 21:13:30.939309 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.939176    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"
Jul 30 21:13:30.939309 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.939216    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk" event={"ID":"ee5fd67c-4d81-48c5-9741-6852192662e7","Type":"ContainerDied","Data":"e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2"}
Jul 30 21:13:30.939309 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.939266    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk" event={"ID":"ee5fd67c-4d81-48c5-9741-6852192662e7","Type":"ContainerDied","Data":"1d98c83a679d40e063882757e3da2219020552e2dd449bee414af2892aaa1cd8"}
Jul 30 21:13:30.947112 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.947088    2547 scope.go:117] "RemoveContainer" containerID="687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb"
Jul 30 21:13:30.947432 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:13:30.947405    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb\": container with ID starting with 687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb not found: ID does not exist" containerID="687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb"
Jul 30 21:13:30.947539 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.947444    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb"} err="failed to get container status \"687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb\": rpc error: code = NotFound desc = could not find container \"687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb\": container with ID starting with 687eaf10695e39dc73e5fbe9027cde862bffc3ad5e8a329aa77221989e0025fb not found: ID does not exist"
Jul 30 21:13:30.947539 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.947468    2547 scope.go:117] "RemoveContainer" containerID="526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35"
Jul 30 21:13:30.956311 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.956177    2547 scope.go:117] "RemoveContainer" containerID="526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35"
Jul 30 21:13:30.956591 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:13:30.956409    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35\": container with ID starting with 526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35 not found: ID does not exist" containerID="526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35"
Jul 30 21:13:30.956591 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.956428    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35"} err="failed to get container status \"526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35\": rpc error: code = NotFound desc = could not find container \"526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35\": container with ID starting with 526788631bd50d66a964acf5cfc4cf0263fcd4adbcc912521e9a15f5ae693c35 not found: ID does not exist"
Jul 30 21:13:30.956591 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.956443    2547 scope.go:117] "RemoveContainer" containerID="e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2"
Jul 30 21:13:30.961870 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.961849    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"]
Jul 30 21:13:30.965864 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.965839    2547 scope.go:117] "RemoveContainer" containerID="e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2"
Jul 30 21:13:30.966171 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:13:30.966137    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2\": container with ID starting with e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2 not found: ID does not exist" containerID="e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2"
Jul 30 21:13:30.966250 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.966172    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2"} err="failed to get container status \"e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2\": rpc error: code = NotFound desc = could not find container \"e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2\": container with ID starting with e0b0c0bc9ed394b59b9e06f5ce7971f60145c374943ae5096a3e22bb47c4c2c2 not found: ID does not exist"
Jul 30 21:13:30.966428 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.966416    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-039a87-65d448bd-5m95v"]
Jul 30 21:13:30.975585 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.975567    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"]
Jul 30 21:13:30.979237 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.979214    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-039a87-6df9759647-5n9ts"]
Jul 30 21:13:30.989157 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.989137    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"]
Jul 30 21:13:30.995326 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:30.995307    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-039a87-66bd668946-zkllk"]
Jul 30 21:13:32.072217 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:32.072188    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0316d4f7-016f-4862-b247-c901dadfbaa6" path="/var/lib/kubelet/pods/0316d4f7-016f-4862-b247-c901dadfbaa6/volumes"
Jul 30 21:13:32.072576 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:32.072538    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc0fb33e-1601-490e-bcd8-b7fa24ecaab5" path="/var/lib/kubelet/pods/cc0fb33e-1601-490e-bcd8-b7fa24ecaab5/volumes"
Jul 30 21:13:32.072925 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:32.072911    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dae5e32b-bfa5-46f8-808d-990e67071836" path="/var/lib/kubelet/pods/dae5e32b-bfa5-46f8-808d-990e67071836/volumes"
Jul 30 21:13:32.073235 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:32.073223    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee5fd67c-4d81-48c5-9741-6852192662e7" path="/var/lib/kubelet/pods/ee5fd67c-4d81-48c5-9741-6852192662e7/volumes"
Jul 30 21:13:41.651660 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.651626    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"]
Jul 30 21:13:41.652319 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652251    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5fd67c-4d81-48c5-9741-6852192662e7" containerName="payload-processing"
Jul 30 21:13:41.652319 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652273    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5fd67c-4d81-48c5-9741-6852192662e7" containerName="payload-processing"
Jul 30 21:13:41.652319 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652290    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0fb33e-1601-490e-bcd8-b7fa24ecaab5" containerName="payload-pre-processing"
Jul 30 21:13:41.652319 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652299    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0fb33e-1601-490e-bcd8-b7fa24ecaab5" containerName="payload-pre-processing"
Jul 30 21:13:41.652544 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652338    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dae5e32b-bfa5-46f8-808d-990e67071836" containerName="maas-api"
Jul 30 21:13:41.652544 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652347    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="dae5e32b-bfa5-46f8-808d-990e67071836" containerName="maas-api"
Jul 30 21:13:41.652544 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652357    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0316d4f7-016f-4862-b247-c901dadfbaa6" containerName="revoke-keys"
Jul 30 21:13:41.652544 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652365    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="0316d4f7-016f-4862-b247-c901dadfbaa6" containerName="revoke-keys"
Jul 30 21:13:41.652544 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652461    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0fb33e-1601-490e-bcd8-b7fa24ecaab5" containerName="payload-pre-processing"
Jul 30 21:13:41.652544 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652473    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="dae5e32b-bfa5-46f8-808d-990e67071836" containerName="maas-api"
Jul 30 21:13:41.652544 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652485    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee5fd67c-4d81-48c5-9741-6852192662e7" containerName="payload-processing"
Jul 30 21:13:41.652544 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.652497    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="0316d4f7-016f-4862-b247-c901dadfbaa6" containerName="revoke-keys"
Jul 30 21:13:41.657508 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.657483    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:41.660097 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.660074    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-155c1d-serving-cert\""
Jul 30 21:13:41.667008 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.666977    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"]
Jul 30 21:13:41.719840 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.719810    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ce84ee-2915-4d16-977d-e29e65cc7fa3-maas-api-tls\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:41.719969 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.719849    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ce84ee-2915-4d16-977d-e29e65cc7fa3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:41.719969 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.719928    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vs95\" (UniqueName: \"kubernetes.io/projected/91ce84ee-2915-4d16-977d-e29e65cc7fa3-kube-api-access-2vs95\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:41.820934 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.820904    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ce84ee-2915-4d16-977d-e29e65cc7fa3-maas-api-tls\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:41.821078 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.820940    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ce84ee-2915-4d16-977d-e29e65cc7fa3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:41.821078 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:13:41.821055    2547 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-serving-cert: secret "maas-api-e2e-shared-a-155c1d-serving-cert" not found
Jul 30 21:13:41.821167 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:13:41.821118    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91ce84ee-2915-4d16-977d-e29e65cc7fa3-maas-api-tls podName:91ce84ee-2915-4d16-977d-e29e65cc7fa3 nodeName:}" failed. No retries permitted until 2026-07-30 21:13:42.321099162 +0000 UTC m=+2900.782057957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91ce84ee-2915-4d16-977d-e29e65cc7fa3-maas-api-tls") pod "maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r" (UID: "91ce84ee-2915-4d16-977d-e29e65cc7fa3") : secret "maas-api-e2e-shared-a-155c1d-serving-cert" not found
Jul 30 21:13:41.821167 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.821136    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vs95\" (UniqueName: \"kubernetes.io/projected/91ce84ee-2915-4d16-977d-e29e65cc7fa3-kube-api-access-2vs95\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:41.821563 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.821544    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ce84ee-2915-4d16-977d-e29e65cc7fa3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:41.829663 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:41.829632    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vs95\" (UniqueName: \"kubernetes.io/projected/91ce84ee-2915-4d16-977d-e29e65cc7fa3-kube-api-access-2vs95\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:42.251935 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.251903    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"]
Jul 30 21:13:42.255551 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.255524    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:42.258060 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.258034    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-155c1d\""
Jul 30 21:13:42.258173 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.258060    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-155c1d-dockercfg-l56h4\""
Jul 30 21:13:42.265588 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.265563    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"]
Jul 30 21:13:42.287580 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.287545    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"]
Jul 30 21:13:42.291413 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.291391    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:13:42.303656 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.303627    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"]
Jul 30 21:13:42.324195 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.324166    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46528b18-2273-4428-8c92-51ed29d88619-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2\" (UID: \"46528b18-2273-4428-8c92-51ed29d88619\") " pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:13:42.324305 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.324211    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft\" (UID: \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:42.324305 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.324271    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w22rf\" (UniqueName: \"kubernetes.io/projected/46528b18-2273-4428-8c92-51ed29d88619-kube-api-access-w22rf\") pod \"payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2\" (UID: \"46528b18-2273-4428-8c92-51ed29d88619\") " pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:13:42.324305 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.324291    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbsfs\" (UniqueName: \"kubernetes.io/projected/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-kube-api-access-qbsfs\") pod \"payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft\" (UID: \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:42.324484 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.324403    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ce84ee-2915-4d16-977d-e29e65cc7fa3-maas-api-tls\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:42.326824 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.326805    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ce84ee-2915-4d16-977d-e29e65cc7fa3-maas-api-tls\") pod \"maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:42.425187 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.425157    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46528b18-2273-4428-8c92-51ed29d88619-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2\" (UID: \"46528b18-2273-4428-8c92-51ed29d88619\") " pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:13:42.425312 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.425200    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft\" (UID: \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:42.425312 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.425229    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w22rf\" (UniqueName: \"kubernetes.io/projected/46528b18-2273-4428-8c92-51ed29d88619-kube-api-access-w22rf\") pod \"payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2\" (UID: \"46528b18-2273-4428-8c92-51ed29d88619\") " pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:13:42.425312 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.425258    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbsfs\" (UniqueName: \"kubernetes.io/projected/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-kube-api-access-qbsfs\") pod \"payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft\" (UID: \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:42.425895 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.425875    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46528b18-2273-4428-8c92-51ed29d88619-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2\" (UID: \"46528b18-2273-4428-8c92-51ed29d88619\") " pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:13:42.425950 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.425939    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft\" (UID: \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:42.433833 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.433811    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbsfs\" (UniqueName: \"kubernetes.io/projected/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-kube-api-access-qbsfs\") pod \"payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft\" (UID: \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:42.433956 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.433864    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w22rf\" (UniqueName: \"kubernetes.io/projected/46528b18-2273-4428-8c92-51ed29d88619-kube-api-access-w22rf\") pod \"payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2\" (UID: \"46528b18-2273-4428-8c92-51ed29d88619\") " pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:13:42.567802 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.567740    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:42.570912 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.570894    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:42.604910 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.604874    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:13:42.727243 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.727218    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"]
Jul 30 21:13:42.729414 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:13:42.729384    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ce84ee_2915_4d16_977d_e29e65cc7fa3.slice/crio-a4803435aebb5b08598ea7975d65e9e32cbc5d511d2db0cd4d152ff1c071adf9 WatchSource:0}: Error finding container a4803435aebb5b08598ea7975d65e9e32cbc5d511d2db0cd4d152ff1c071adf9: Status 404 returned error can't find the container with id a4803435aebb5b08598ea7975d65e9e32cbc5d511d2db0cd4d152ff1c071adf9
Jul 30 21:13:42.762005 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.761980    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"]
Jul 30 21:13:42.763905 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:13:42.763863    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc1d985_0323_4b86_a3e9_26f83ba2b02a.slice/crio-67f11c8ae5d02180649e60242f0e61b2d99533a563217b15366ae53769ff94b5 WatchSource:0}: Error finding container 67f11c8ae5d02180649e60242f0e61b2d99533a563217b15366ae53769ff94b5: Status 404 returned error can't find the container with id 67f11c8ae5d02180649e60242f0e61b2d99533a563217b15366ae53769ff94b5
Jul 30 21:13:42.790250 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:42.790226    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"]
Jul 30 21:13:42.803789 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:13:42.803761    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46528b18_2273_4428_8c92_51ed29d88619.slice/crio-c30b8b8f2e2061e17c7ad715326815d4d2b13e25ece36b0859366621eccab5ff WatchSource:0}: Error finding container c30b8b8f2e2061e17c7ad715326815d4d2b13e25ece36b0859366621eccab5ff: Status 404 returned error can't find the container with id c30b8b8f2e2061e17c7ad715326815d4d2b13e25ece36b0859366621eccab5ff
Jul 30 21:13:43.017252 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:43.017211    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft" event={"ID":"bdc1d985-0323-4b86-a3e9-26f83ba2b02a","Type":"ContainerStarted","Data":"33c6abb9016aca4acdc36ea08225085a4fd1642656287d4aaaf522222b7ce9f5"}
Jul 30 21:13:43.017443 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:43.017281    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:43.017443 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:43.017308    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft" event={"ID":"bdc1d985-0323-4b86-a3e9-26f83ba2b02a","Type":"ContainerStarted","Data":"67f11c8ae5d02180649e60242f0e61b2d99533a563217b15366ae53769ff94b5"}
Jul 30 21:13:43.018684 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:43.018651    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r" event={"ID":"91ce84ee-2915-4d16-977d-e29e65cc7fa3","Type":"ContainerStarted","Data":"a4803435aebb5b08598ea7975d65e9e32cbc5d511d2db0cd4d152ff1c071adf9"}
Jul 30 21:13:43.020179 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:43.020147    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2" event={"ID":"46528b18-2273-4428-8c92-51ed29d88619","Type":"ContainerStarted","Data":"f4a619caf7f7d6259401fffea7ef1f25c8d1d03a93fd834b740147c6bcc89a0c"}
Jul 30 21:13:43.020272 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:43.020189    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2" event={"ID":"46528b18-2273-4428-8c92-51ed29d88619","Type":"ContainerStarted","Data":"c30b8b8f2e2061e17c7ad715326815d4d2b13e25ece36b0859366621eccab5ff"}
Jul 30 21:13:43.020330 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:43.020312    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:13:43.033385 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:43.033346    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft" podStartSLOduration=1.033333873 podStartE2EDuration="1.033333873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:13:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:13:43.032497103 +0000 UTC m=+2901.493455914" watchObservedRunningTime="2026-07-30 21:13:43.033333873 +0000 UTC m=+2901.494292685"
Jul 30 21:13:43.048569 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:43.048533    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2" podStartSLOduration=1.048517119 podStartE2EDuration="1.048517119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:13:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:13:43.047283078 +0000 UTC m=+2901.508241894" watchObservedRunningTime="2026-07-30 21:13:43.048517119 +0000 UTC m=+2901.509475930"
Jul 30 21:13:44.026409 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:44.026371    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r" event={"ID":"91ce84ee-2915-4d16-977d-e29e65cc7fa3","Type":"ContainerStarted","Data":"3d9d2f377a71dfab0b61b4fea729138519cae8700b1246f3e2c0c6616c9a2589"}
Jul 30 21:13:44.026885 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:44.026457    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:44.047390 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:44.047347    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r" podStartSLOduration=2.522189921 podStartE2EDuration="3.047335159s" totalImagesPullingTime="525.145238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:13:44.047172359 +0000 UTC m=+2902.508131161" watchObservedRunningTime="2026-07-30 21:13:44.047335159 +0000 UTC m=+2902.508293973"
Jul 30 21:13:50.037204 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:50.037165    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:13:54.028930 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:54.028891    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:13:54.029344 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:13:54.029074    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:14:09.740525 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.740488    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"]
Jul 30 21:14:09.744633 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.744612    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:09.747253 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.747233    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-01bc1b-serving-cert\""
Jul 30 21:14:09.756204 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.756176    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"]
Jul 30 21:14:09.846765 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.846739    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcn4c\" (UniqueName: \"kubernetes.io/projected/57f30a57-067a-4dfe-9142-b71b137a53db-kube-api-access-pcn4c\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:09.846890 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.846841    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57f30a57-067a-4dfe-9142-b71b137a53db-openshift-service-ca\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:09.846890 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.846869    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57f30a57-067a-4dfe-9142-b71b137a53db-maas-api-tls\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:09.947992 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.947965    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57f30a57-067a-4dfe-9142-b71b137a53db-openshift-service-ca\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:09.948093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.947996    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57f30a57-067a-4dfe-9142-b71b137a53db-maas-api-tls\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:09.948093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.948030    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcn4c\" (UniqueName: \"kubernetes.io/projected/57f30a57-067a-4dfe-9142-b71b137a53db-kube-api-access-pcn4c\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:09.948176 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:14:09.948107    2547 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-serving-cert: secret "maas-api-e2e-shared-b-01bc1b-serving-cert" not found
Jul 30 21:14:09.948176 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:14:09.948172    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57f30a57-067a-4dfe-9142-b71b137a53db-maas-api-tls podName:57f30a57-067a-4dfe-9142-b71b137a53db nodeName:}" failed. No retries permitted until 2026-07-30 21:14:10.448157951 +0000 UTC m=+2928.909116742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/57f30a57-067a-4dfe-9142-b71b137a53db-maas-api-tls") pod "maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2" (UID: "57f30a57-067a-4dfe-9142-b71b137a53db") : secret "maas-api-e2e-shared-b-01bc1b-serving-cert" not found
Jul 30 21:14:09.948589 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.948571    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57f30a57-067a-4dfe-9142-b71b137a53db-openshift-service-ca\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:09.956473 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:09.956457    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcn4c\" (UniqueName: \"kubernetes.io/projected/57f30a57-067a-4dfe-9142-b71b137a53db-kube-api-access-pcn4c\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:10.402945 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.402902    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"]
Jul 30 21:14:10.406645 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.406621    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:10.409200 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.409176    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-01bc1b\""
Jul 30 21:14:10.409366 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.409347    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-01bc1b-dockercfg-6czqc\""
Jul 30 21:14:10.417277 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.417254    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"]
Jul 30 21:14:10.430181 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.430162    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"]
Jul 30 21:14:10.439837 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.439807    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:14:10.444799 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.444776    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"]
Jul 30 21:14:10.452209 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.452180    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57f30a57-067a-4dfe-9142-b71b137a53db-maas-api-tls\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:10.455243 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.455221    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57f30a57-067a-4dfe-9142-b71b137a53db-maas-api-tls\") pod \"maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:10.553004 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.552981    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afad7662-a865-4a3c-813b-da51986a06d7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k\" (UID: \"afad7662-a865-4a3c-813b-da51986a06d7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:10.553124 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.553013    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3c07009-f54d-40fb-8607-0adb22f99e1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2\" (UID: \"e3c07009-f54d-40fb-8607-0adb22f99e1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:14:10.553124 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.553033    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2kd7\" (UniqueName: \"kubernetes.io/projected/e3c07009-f54d-40fb-8607-0adb22f99e1c-kube-api-access-b2kd7\") pod \"payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2\" (UID: \"e3c07009-f54d-40fb-8607-0adb22f99e1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:14:10.553124 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.553101    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ktsd\" (UniqueName: \"kubernetes.io/projected/afad7662-a865-4a3c-813b-da51986a06d7-kube-api-access-6ktsd\") pod \"payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k\" (UID: \"afad7662-a865-4a3c-813b-da51986a06d7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:10.654180 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.654120    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afad7662-a865-4a3c-813b-da51986a06d7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k\" (UID: \"afad7662-a865-4a3c-813b-da51986a06d7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:10.654180 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.654156    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3c07009-f54d-40fb-8607-0adb22f99e1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2\" (UID: \"e3c07009-f54d-40fb-8607-0adb22f99e1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:14:10.654314 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.654182    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2kd7\" (UniqueName: \"kubernetes.io/projected/e3c07009-f54d-40fb-8607-0adb22f99e1c-kube-api-access-b2kd7\") pod \"payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2\" (UID: \"e3c07009-f54d-40fb-8607-0adb22f99e1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:14:10.654314 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.654224    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ktsd\" (UniqueName: \"kubernetes.io/projected/afad7662-a865-4a3c-813b-da51986a06d7-kube-api-access-6ktsd\") pod \"payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k\" (UID: \"afad7662-a865-4a3c-813b-da51986a06d7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:10.654883 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.654865    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afad7662-a865-4a3c-813b-da51986a06d7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k\" (UID: \"afad7662-a865-4a3c-813b-da51986a06d7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:10.654929 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.654912    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3c07009-f54d-40fb-8607-0adb22f99e1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2\" (UID: \"e3c07009-f54d-40fb-8607-0adb22f99e1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:14:10.657099 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.657083    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:10.662123 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.662098    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ktsd\" (UniqueName: \"kubernetes.io/projected/afad7662-a865-4a3c-813b-da51986a06d7-kube-api-access-6ktsd\") pod \"payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k\" (UID: \"afad7662-a865-4a3c-813b-da51986a06d7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:10.662222 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.662160    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2kd7\" (UniqueName: \"kubernetes.io/projected/e3c07009-f54d-40fb-8607-0adb22f99e1c-kube-api-access-b2kd7\") pod \"payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2\" (UID: \"e3c07009-f54d-40fb-8607-0adb22f99e1c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:14:10.718085 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.718053    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:10.755493 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.755237    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:14:10.810969 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.810901    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"]
Jul 30 21:14:10.812356 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:14:10.812314    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f30a57_067a_4dfe_9142_b71b137a53db.slice/crio-1b5496cd2882ab461b34669ede199fcebaa8d9e1f682629706bd37e4eb32f95f WatchSource:0}: Error finding container 1b5496cd2882ab461b34669ede199fcebaa8d9e1f682629706bd37e4eb32f95f: Status 404 returned error can't find the container with id 1b5496cd2882ab461b34669ede199fcebaa8d9e1f682629706bd37e4eb32f95f
Jul 30 21:14:10.871271 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:10.871243    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"]
Jul 30 21:14:10.874847 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:14:10.874820    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafad7662_a865_4a3c_813b_da51986a06d7.slice/crio-7ebb8e573213c68dabdb64f8b6b2224403f34f4483a646f84f0eeb997c0fbb80 WatchSource:0}: Error finding container 7ebb8e573213c68dabdb64f8b6b2224403f34f4483a646f84f0eeb997c0fbb80: Status 404 returned error can't find the container with id 7ebb8e573213c68dabdb64f8b6b2224403f34f4483a646f84f0eeb997c0fbb80
Jul 30 21:14:11.120399 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:11.120364    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"]
Jul 30 21:14:11.122558 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:14:11.122536    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c07009_f54d_40fb_8607_0adb22f99e1c.slice/crio-129a39df58498c6e9f6fe022d8afcd874fffc17ff553fe20fdc282ea60f9605b WatchSource:0}: Error finding container 129a39df58498c6e9f6fe022d8afcd874fffc17ff553fe20fdc282ea60f9605b: Status 404 returned error can't find the container with id 129a39df58498c6e9f6fe022d8afcd874fffc17ff553fe20fdc282ea60f9605b
Jul 30 21:14:11.167079 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:11.167021    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2" event={"ID":"e3c07009-f54d-40fb-8607-0adb22f99e1c","Type":"ContainerStarted","Data":"129a39df58498c6e9f6fe022d8afcd874fffc17ff553fe20fdc282ea60f9605b"}
Jul 30 21:14:11.168712 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:11.168673    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k" event={"ID":"afad7662-a865-4a3c-813b-da51986a06d7","Type":"ContainerStarted","Data":"a2e59a30a2e4ef573a4089c059fe60329ffa869d64f313c0b0b3cec50af89ba1"}
Jul 30 21:14:11.168818 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:11.168754    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:11.168818 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:11.168776    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k" event={"ID":"afad7662-a865-4a3c-813b-da51986a06d7","Type":"ContainerStarted","Data":"7ebb8e573213c68dabdb64f8b6b2224403f34f4483a646f84f0eeb997c0fbb80"}
Jul 30 21:14:11.169908 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:11.169888    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2" event={"ID":"57f30a57-067a-4dfe-9142-b71b137a53db","Type":"ContainerStarted","Data":"1b5496cd2882ab461b34669ede199fcebaa8d9e1f682629706bd37e4eb32f95f"}
Jul 30 21:14:11.186585 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:11.186549    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k" podStartSLOduration=1.186536473 podStartE2EDuration="1.186536473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:14:11.184607924 +0000 UTC m=+2929.645566735" watchObservedRunningTime="2026-07-30 21:14:11.186536473 +0000 UTC m=+2929.647495283"
Jul 30 21:14:12.177196 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:12.177158    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2" event={"ID":"e3c07009-f54d-40fb-8607-0adb22f99e1c","Type":"ContainerStarted","Data":"a9c589ae08c227cc8c530e2b01f1979de18304c61e772e929be28bb8dcf9aa24"}
Jul 30 21:14:12.177614 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:12.177215    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:14:12.178763 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:12.178743    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2" event={"ID":"57f30a57-067a-4dfe-9142-b71b137a53db","Type":"ContainerStarted","Data":"2dec810def139848cf185f16e0f0955d385379db50111a614ae92bec218acbda"}
Jul 30 21:14:12.178831 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:12.178796    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:12.194493 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:12.194443    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2" podStartSLOduration=2.194426968 podStartE2EDuration="2.194426968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:14:12.193531285 +0000 UTC m=+2930.654490099" watchObservedRunningTime="2026-07-30 21:14:12.194426968 +0000 UTC m=+2930.655385782"
Jul 30 21:14:12.209103 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:12.209055    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2" podStartSLOduration=2.819809593 podStartE2EDuration="3.209038184s" totalImagesPullingTime="389.228591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:14:12.208483723 +0000 UTC m=+2930.669442533" watchObservedRunningTime="2026-07-30 21:14:12.209038184 +0000 UTC m=+2930.669997000"
Jul 30 21:14:18.188835 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:18.188807    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:14:22.180791 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:22.180763    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:14:23.185851 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:14:23.185821    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:15:22.254837 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:15:22.254809    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:15:22.259121 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:15:22.259099    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:15:22.266652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:15:22.266633    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:15:22.270432 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:15:22.270415    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:19:22.434801 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:19:22.434763    2547 scope.go:117] "RemoveContainer" containerID="3b819939bf262c69a53b8217fa4e58aa053212e5e9f6faa1f13acb68086eb248"
Jul 30 21:20:22.284219 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:20:22.284195    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:20:22.288197 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:20:22.288177    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:20:22.299382 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:20:22.299364    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:20:22.303250 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:20:22.303231    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:23:05.449625 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.449592    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"]
Jul 30 21:23:05.453096 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.453077    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:05.455652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.455633    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-e54afd-serving-cert\""
Jul 30 21:23:05.467628 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.467594    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"]
Jul 30 21:23:05.522522 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.522500    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d54db2-5dcf-4221-81f2-48219f3ef989-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:05.522653 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.522566    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzqhv\" (UniqueName: \"kubernetes.io/projected/b6d54db2-5dcf-4221-81f2-48219f3ef989-kube-api-access-kzqhv\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:05.522755 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.522663    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d54db2-5dcf-4221-81f2-48219f3ef989-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:05.623949 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.623921    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzqhv\" (UniqueName: \"kubernetes.io/projected/b6d54db2-5dcf-4221-81f2-48219f3ef989-kube-api-access-kzqhv\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:05.624075 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.623966    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d54db2-5dcf-4221-81f2-48219f3ef989-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:05.624075 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.624006    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d54db2-5dcf-4221-81f2-48219f3ef989-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:05.624191 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:23:05.624085    2547 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-serving-cert: secret "maas-api-e2e-ipp-a-e54afd-serving-cert" not found
Jul 30 21:23:05.624191 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:23:05.624147    2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d54db2-5dcf-4221-81f2-48219f3ef989-maas-api-tls podName:b6d54db2-5dcf-4221-81f2-48219f3ef989 nodeName:}" failed. No retries permitted until 2026-07-30 21:23:06.124129057 +0000 UTC m=+3464.585087847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6d54db2-5dcf-4221-81f2-48219f3ef989-maas-api-tls") pod "maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj" (UID: "b6d54db2-5dcf-4221-81f2-48219f3ef989") : secret "maas-api-e2e-ipp-a-e54afd-serving-cert" not found
Jul 30 21:23:05.624616 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.624596    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d54db2-5dcf-4221-81f2-48219f3ef989-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:05.634639 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:05.634615    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzqhv\" (UniqueName: \"kubernetes.io/projected/b6d54db2-5dcf-4221-81f2-48219f3ef989-kube-api-access-kzqhv\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:06.128076 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.128038    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d54db2-5dcf-4221-81f2-48219f3ef989-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:06.130620 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.130596    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d54db2-5dcf-4221-81f2-48219f3ef989-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:06.166311 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.166280    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"]
Jul 30 21:23:06.170102 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.170082    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:06.172772 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.172754    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-e54afd\""
Jul 30 21:23:06.172966 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.172948    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-e54afd-dockercfg-862nf\""
Jul 30 21:23:06.179383 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.179361    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"]
Jul 30 21:23:06.229230 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.229202    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a0caf6-2b40-4d4d-907f-0558af558f5d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h\" (UID: \"10a0caf6-2b40-4d4d-907f-0558af558f5d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:06.229403 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.229383    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgk5d\" (UniqueName: \"kubernetes.io/projected/10a0caf6-2b40-4d4d-907f-0558af558f5d-kube-api-access-bgk5d\") pod \"payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h\" (UID: \"10a0caf6-2b40-4d4d-907f-0558af558f5d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:06.329972 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.329942    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a0caf6-2b40-4d4d-907f-0558af558f5d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h\" (UID: \"10a0caf6-2b40-4d4d-907f-0558af558f5d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:06.330117 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.330071    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgk5d\" (UniqueName: \"kubernetes.io/projected/10a0caf6-2b40-4d4d-907f-0558af558f5d-kube-api-access-bgk5d\") pod \"payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h\" (UID: \"10a0caf6-2b40-4d4d-907f-0558af558f5d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:06.330615 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.330592    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a0caf6-2b40-4d4d-907f-0558af558f5d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h\" (UID: \"10a0caf6-2b40-4d4d-907f-0558af558f5d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:06.338262 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.338235    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgk5d\" (UniqueName: \"kubernetes.io/projected/10a0caf6-2b40-4d4d-907f-0558af558f5d-kube-api-access-bgk5d\") pod \"payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h\" (UID: \"10a0caf6-2b40-4d4d-907f-0558af558f5d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:06.363250 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.363222    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:06.483017 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.482988    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:06.498214 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.498191    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"]
Jul 30 21:23:06.500368 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:23:06.500341    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d54db2_5dcf_4221_81f2_48219f3ef989.slice/crio-9fab13ac9aae070c7026122eb38bb4dafcc49e16a1994ecd14fa1b1f2457d464 WatchSource:0}: Error finding container 9fab13ac9aae070c7026122eb38bb4dafcc49e16a1994ecd14fa1b1f2457d464: Status 404 returned error can't find the container with id 9fab13ac9aae070c7026122eb38bb4dafcc49e16a1994ecd14fa1b1f2457d464
Jul 30 21:23:06.501955 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.501935    2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 21:23:06.616954 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.616924    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"]
Jul 30 21:23:06.618780 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:23:06.618754    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a0caf6_2b40_4d4d_907f_0558af558f5d.slice/crio-c5200d2cf56e3e920f5f6b6a239193a60bfbe92b50dbaedfde7774eb5fd7497f WatchSource:0}: Error finding container c5200d2cf56e3e920f5f6b6a239193a60bfbe92b50dbaedfde7774eb5fd7497f: Status 404 returned error can't find the container with id c5200d2cf56e3e920f5f6b6a239193a60bfbe92b50dbaedfde7774eb5fd7497f
Jul 30 21:23:06.928005 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.927929    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h" event={"ID":"10a0caf6-2b40-4d4d-907f-0558af558f5d","Type":"ContainerStarted","Data":"11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376"}
Jul 30 21:23:06.928005 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.927993    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:06.928220 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.928013    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h" event={"ID":"10a0caf6-2b40-4d4d-907f-0558af558f5d","Type":"ContainerStarted","Data":"c5200d2cf56e3e920f5f6b6a239193a60bfbe92b50dbaedfde7774eb5fd7497f"}
Jul 30 21:23:06.929239 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.929211    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj" event={"ID":"b6d54db2-5dcf-4221-81f2-48219f3ef989","Type":"ContainerStarted","Data":"9fab13ac9aae070c7026122eb38bb4dafcc49e16a1994ecd14fa1b1f2457d464"}
Jul 30 21:23:06.944006 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:06.943961    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h" podStartSLOduration=0.943946185 podStartE2EDuration="943.946185ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:23:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:23:06.943419001 +0000 UTC m=+3465.404377811" watchObservedRunningTime="2026-07-30 21:23:06.943946185 +0000 UTC m=+3465.404904978"
Jul 30 21:23:07.936452 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:07.936405    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj" event={"ID":"b6d54db2-5dcf-4221-81f2-48219f3ef989","Type":"ContainerStarted","Data":"eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4"}
Jul 30 21:23:07.936880 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:07.936686    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:07.954877 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:07.954831    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj" podStartSLOduration=2.52382317 podStartE2EDuration="2.95481734s" totalImagesPullingTime="430.99417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:23:07.953790195 +0000 UTC m=+3466.414749008" watchObservedRunningTime="2026-07-30 21:23:07.95481734 +0000 UTC m=+3466.415776199"
Jul 30 21:23:13.947958 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:13.947927    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:23:17.938800 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:17.938769    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:23:37.359507 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.359473    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"]
Jul 30 21:23:37.362870 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.362851    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.366123 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.366104    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-296998-serving-cert\""
Jul 30 21:23:37.375444 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.375416    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"]
Jul 30 21:23:37.485138 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.485105    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd8sd\" (UniqueName: \"kubernetes.io/projected/c45c325e-03a5-4290-a1f1-b35bacf83425-kube-api-access-vd8sd\") pod \"maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.485294 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.485234    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c45c325e-03a5-4290-a1f1-b35bacf83425-maas-api-tls\") pod \"maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.485294 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.485281    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45c325e-03a5-4290-a1f1-b35bacf83425-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.586053 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.586026    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd8sd\" (UniqueName: \"kubernetes.io/projected/c45c325e-03a5-4290-a1f1-b35bacf83425-kube-api-access-vd8sd\") pod \"maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.586172 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.586092    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c45c325e-03a5-4290-a1f1-b35bacf83425-maas-api-tls\") pod \"maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.586172 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.586131    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45c325e-03a5-4290-a1f1-b35bacf83425-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.586824 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.586799    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45c325e-03a5-4290-a1f1-b35bacf83425-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.588688 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.588661    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c45c325e-03a5-4290-a1f1-b35bacf83425-maas-api-tls\") pod \"maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.597687 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.597664    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd8sd\" (UniqueName: \"kubernetes.io/projected/c45c325e-03a5-4290-a1f1-b35bacf83425-kube-api-access-vd8sd\") pod \"maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.672923 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.672869    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:37.820252 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:37.820228    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"]
Jul 30 21:23:37.822599 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:23:37.822569    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45c325e_03a5_4290_a1f1_b35bacf83425.slice/crio-3da423f652740e097f239e5693c33a78fd5629349a7d07ff4daa224a6ae1210d WatchSource:0}: Error finding container 3da423f652740e097f239e5693c33a78fd5629349a7d07ff4daa224a6ae1210d: Status 404 returned error can't find the container with id 3da423f652740e097f239e5693c33a78fd5629349a7d07ff4daa224a6ae1210d
Jul 30 21:23:38.088076 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:38.088047    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v" event={"ID":"c45c325e-03a5-4290-a1f1-b35bacf83425","Type":"ContainerStarted","Data":"3da423f652740e097f239e5693c33a78fd5629349a7d07ff4daa224a6ae1210d"}
Jul 30 21:23:39.095056 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:39.095021    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v" event={"ID":"c45c325e-03a5-4290-a1f1-b35bacf83425","Type":"ContainerStarted","Data":"53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8"}
Jul 30 21:23:39.095473 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:39.095088    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:23:39.159482 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:39.159408    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v" podStartSLOduration=1.780463299 podStartE2EDuration="2.159394019s" totalImagesPullingTime="378.93072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:23:39.157586269 +0000 UTC m=+3497.618545079" watchObservedRunningTime="2026-07-30 21:23:39.159394019 +0000 UTC m=+3497.620352830"
Jul 30 21:23:45.105177 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:23:45.105094    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:25:08.614612 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:08.614572    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h_10a0caf6-2b40-4d4d-907f-0558af558f5d/payload-processing/0.log"
Jul 30 21:25:14.020890 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.020809    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"]
Jul 30 21:25:14.024788 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.024767    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.027791 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.027770    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-83307c-serving-cert\""
Jul 30 21:25:14.038449 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.038425    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"]
Jul 30 21:25:14.152034 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.152005    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3611b816-3169-4f44-a76c-f2e631a1fa37-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.152169 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.152052    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3611b816-3169-4f44-a76c-f2e631a1fa37-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.152169 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.152089    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvtr8\" (UniqueName: \"kubernetes.io/projected/3611b816-3169-4f44-a76c-f2e631a1fa37-kube-api-access-fvtr8\") pod \"maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.252620 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.252587    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3611b816-3169-4f44-a76c-f2e631a1fa37-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.252620 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.252632    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvtr8\" (UniqueName: \"kubernetes.io/projected/3611b816-3169-4f44-a76c-f2e631a1fa37-kube-api-access-fvtr8\") pod \"maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.252928 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.252903    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3611b816-3169-4f44-a76c-f2e631a1fa37-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.253394 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.253369    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3611b816-3169-4f44-a76c-f2e631a1fa37-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.255338 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.255318    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3611b816-3169-4f44-a76c-f2e631a1fa37-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.270982 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.270925    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvtr8\" (UniqueName: \"kubernetes.io/projected/3611b816-3169-4f44-a76c-f2e631a1fa37-kube-api-access-fvtr8\") pod \"maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.335771 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.335746    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:14.686792 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.686768    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"]
Jul 30 21:25:14.689122 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:25:14.689096    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3611b816_3169_4f44_a76c_f2e631a1fa37.slice/crio-14161722c9344bd9a78204dbc192f0c86fe26a97924101717c88dde11776e2b3 WatchSource:0}: Error finding container 14161722c9344bd9a78204dbc192f0c86fe26a97924101717c88dde11776e2b3: Status 404 returned error can't find the container with id 14161722c9344bd9a78204dbc192f0c86fe26a97924101717c88dde11776e2b3
Jul 30 21:25:14.919321 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.919293    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"]
Jul 30 21:25:14.924332 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.924316    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:14.927178 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.927139    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-83307c\""
Jul 30 21:25:14.927329 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.927311    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-83307c-dockercfg-h67dt\""
Jul 30 21:25:14.937389 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:14.937335    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"]
Jul 30 21:25:15.013825 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.013794    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"]
Jul 30 21:25:15.017404 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.017386    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:15.033470 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.033409    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"]
Jul 30 21:25:15.061162 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.061141    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph\" (UID: \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:15.061302 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.061258    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p8s8\" (UniqueName: \"kubernetes.io/projected/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-kube-api-access-5p8s8\") pod \"payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph\" (UID: \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:15.162335 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.162303    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8s8\" (UniqueName: \"kubernetes.io/projected/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-kube-api-access-5p8s8\") pod \"payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph\" (UID: \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:15.162469 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.162357    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63c2c76f-66da-49c3-88f9-c474016ebd10-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7\" (UID: \"63c2c76f-66da-49c3-88f9-c474016ebd10\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:15.162469 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.162430    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqwhw\" (UniqueName: \"kubernetes.io/projected/63c2c76f-66da-49c3-88f9-c474016ebd10-kube-api-access-jqwhw\") pod \"payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7\" (UID: \"63c2c76f-66da-49c3-88f9-c474016ebd10\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:15.162469 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.162460    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph\" (UID: \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:15.163230 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.163207    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph\" (UID: \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:15.173870 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.173851    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p8s8\" (UniqueName: \"kubernetes.io/projected/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-kube-api-access-5p8s8\") pod \"payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph\" (UID: \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:15.236099 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.236077    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:15.263839 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.263809    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqwhw\" (UniqueName: \"kubernetes.io/projected/63c2c76f-66da-49c3-88f9-c474016ebd10-kube-api-access-jqwhw\") pod \"payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7\" (UID: \"63c2c76f-66da-49c3-88f9-c474016ebd10\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:15.264033 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.264012    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63c2c76f-66da-49c3-88f9-c474016ebd10-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7\" (UID: \"63c2c76f-66da-49c3-88f9-c474016ebd10\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:15.265181 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.265151    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63c2c76f-66da-49c3-88f9-c474016ebd10-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7\" (UID: \"63c2c76f-66da-49c3-88f9-c474016ebd10\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:15.273284 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.273261    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqwhw\" (UniqueName: \"kubernetes.io/projected/63c2c76f-66da-49c3-88f9-c474016ebd10-kube-api-access-jqwhw\") pod \"payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7\" (UID: \"63c2c76f-66da-49c3-88f9-c474016ebd10\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:15.328058 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.327990    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:15.582694 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.582668    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"]
Jul 30 21:25:15.584522 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:25:15.584495    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb15a340_07ce_48f1_b9f4_f512a34bd3a0.slice/crio-30d85409c47e19e40145dac67aa0b259899a4144ffaf60f380c39cb056d7990d WatchSource:0}: Error finding container 30d85409c47e19e40145dac67aa0b259899a4144ffaf60f380c39cb056d7990d: Status 404 returned error can't find the container with id 30d85409c47e19e40145dac67aa0b259899a4144ffaf60f380c39cb056d7990d
Jul 30 21:25:15.589236 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.589213    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7" event={"ID":"3611b816-3169-4f44-a76c-f2e631a1fa37","Type":"ContainerStarted","Data":"94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488"}
Jul 30 21:25:15.589325 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.589259    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7" event={"ID":"3611b816-3169-4f44-a76c-f2e631a1fa37","Type":"ContainerStarted","Data":"14161722c9344bd9a78204dbc192f0c86fe26a97924101717c88dde11776e2b3"}
Jul 30 21:25:15.591780 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.591760    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:15.616201 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.616158    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7" podStartSLOduration=1.12194396 podStartE2EDuration="1.616143389s" totalImagesPullingTime="494.199429ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:25:15.614082528 +0000 UTC m=+3594.075041339" watchObservedRunningTime="2026-07-30 21:25:15.616143389 +0000 UTC m=+3594.077102198"
Jul 30 21:25:15.680978 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:15.680956    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"]
Jul 30 21:25:15.682101 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:25:15.682080    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c2c76f_66da_49c3_88f9_c474016ebd10.slice/crio-a72dc5047aecd4ed9f62deacf81cb2bd7993a1b95538dd96e3e3abbf4697a415 WatchSource:0}: Error finding container a72dc5047aecd4ed9f62deacf81cb2bd7993a1b95538dd96e3e3abbf4697a415: Status 404 returned error can't find the container with id a72dc5047aecd4ed9f62deacf81cb2bd7993a1b95538dd96e3e3abbf4697a415
Jul 30 21:25:16.600204 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:16.600162    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph" event={"ID":"fb15a340-07ce-48f1-b9f4-f512a34bd3a0","Type":"ContainerStarted","Data":"f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b"}
Jul 30 21:25:16.600679 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:16.600223    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph" event={"ID":"fb15a340-07ce-48f1-b9f4-f512a34bd3a0","Type":"ContainerStarted","Data":"30d85409c47e19e40145dac67aa0b259899a4144ffaf60f380c39cb056d7990d"}
Jul 30 21:25:16.600679 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:16.600356    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:16.602086 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:16.602060    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7" event={"ID":"63c2c76f-66da-49c3-88f9-c474016ebd10","Type":"ContainerStarted","Data":"49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb"}
Jul 30 21:25:16.602240 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:16.602098    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7" event={"ID":"63c2c76f-66da-49c3-88f9-c474016ebd10","Type":"ContainerStarted","Data":"a72dc5047aecd4ed9f62deacf81cb2bd7993a1b95538dd96e3e3abbf4697a415"}
Jul 30 21:25:16.602240 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:16.602154    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:16.619668 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:16.619627    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph" podStartSLOduration=2.61961391 podStartE2EDuration="2.61961391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:25:16.616819105 +0000 UTC m=+3595.077777917" watchObservedRunningTime="2026-07-30 21:25:16.61961391 +0000 UTC m=+3595.080572759"
Jul 30 21:25:16.635352 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:16.635309    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7" podStartSLOduration=2.635292407 podStartE2EDuration="2.635292407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:25:16.633217508 +0000 UTC m=+3595.094176320" watchObservedRunningTime="2026-07-30 21:25:16.635292407 +0000 UTC m=+3595.096251222"
Jul 30 21:25:21.607918 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:21.607890    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:22.317881 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:22.317856    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:25:22.321747 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:22.321724    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:25:22.334100 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:22.334078    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:25:22.337587 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:22.337567    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-clxfr_9bdc8886-b53f-4145-ac66-ac4de098e6e4/ovn-acl-logging/0.log"
Jul 30 21:25:27.610031 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:27.610001    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:27.610490 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:27.610079    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:44.698232 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:44.698198    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"]
Jul 30 21:25:44.698805 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:44.698433    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7" podUID="3611b816-3169-4f44-a76c-f2e631a1fa37" containerName="maas-api" containerID="cri-o://94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488" gracePeriod=30
Jul 30 21:25:44.768481 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:44.768454    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"]
Jul 30 21:25:44.768720 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:44.768660    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7" podUID="63c2c76f-66da-49c3-88f9-c474016ebd10" containerName="payload-processing" containerID="cri-o://49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb" gracePeriod=30
Jul 30 21:25:44.774500 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:44.774473    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"]
Jul 30 21:25:44.774749 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:44.774718    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph" podUID="fb15a340-07ce-48f1-b9f4-f512a34bd3a0" containerName="payload-pre-processing" containerID="cri-o://f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b" gracePeriod=30
Jul 30 21:25:45.045464 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.045435    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:45.074833 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.074810    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:45.094478 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.094458    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:45.120195 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.119677    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqwhw\" (UniqueName: \"kubernetes.io/projected/63c2c76f-66da-49c3-88f9-c474016ebd10-kube-api-access-jqwhw\") pod \"63c2c76f-66da-49c3-88f9-c474016ebd10\" (UID: \"63c2c76f-66da-49c3-88f9-c474016ebd10\") "
Jul 30 21:25:45.120195 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.119830    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63c2c76f-66da-49c3-88f9-c474016ebd10-plugins-config-volume\") pod \"63c2c76f-66da-49c3-88f9-c474016ebd10\" (UID: \"63c2c76f-66da-49c3-88f9-c474016ebd10\") "
Jul 30 21:25:45.120413 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.120391    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63c2c76f-66da-49c3-88f9-c474016ebd10-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "63c2c76f-66da-49c3-88f9-c474016ebd10" (UID: "63c2c76f-66da-49c3-88f9-c474016ebd10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:25:45.122373 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.122349    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c2c76f-66da-49c3-88f9-c474016ebd10-kube-api-access-jqwhw" (OuterVolumeSpecName: "kube-api-access-jqwhw") pod "63c2c76f-66da-49c3-88f9-c474016ebd10" (UID: "63c2c76f-66da-49c3-88f9-c474016ebd10"). InnerVolumeSpecName "kube-api-access-jqwhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:25:45.221239 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.221214    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-plugins-config-volume\") pod \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\" (UID: \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\") "
Jul 30 21:25:45.221351 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.221279    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3611b816-3169-4f44-a76c-f2e631a1fa37-openshift-service-ca\") pod \"3611b816-3169-4f44-a76c-f2e631a1fa37\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") "
Jul 30 21:25:45.221351 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.221338    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3611b816-3169-4f44-a76c-f2e631a1fa37-maas-api-tls\") pod \"3611b816-3169-4f44-a76c-f2e631a1fa37\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") "
Jul 30 21:25:45.221462 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.221355    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p8s8\" (UniqueName: \"kubernetes.io/projected/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-kube-api-access-5p8s8\") pod \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\" (UID: \"fb15a340-07ce-48f1-b9f4-f512a34bd3a0\") "
Jul 30 21:25:45.221462 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.221388    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvtr8\" (UniqueName: \"kubernetes.io/projected/3611b816-3169-4f44-a76c-f2e631a1fa37-kube-api-access-fvtr8\") pod \"3611b816-3169-4f44-a76c-f2e631a1fa37\" (UID: \"3611b816-3169-4f44-a76c-f2e631a1fa37\") "
Jul 30 21:25:45.221639 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.221620    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqwhw\" (UniqueName: \"kubernetes.io/projected/63c2c76f-66da-49c3-88f9-c474016ebd10-kube-api-access-jqwhw\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:25:45.221730 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.221652    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63c2c76f-66da-49c3-88f9-c474016ebd10-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:25:45.221730 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.221618    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fb15a340-07ce-48f1-b9f4-f512a34bd3a0" (UID: "fb15a340-07ce-48f1-b9f4-f512a34bd3a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:25:45.221730 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.221685    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3611b816-3169-4f44-a76c-f2e631a1fa37-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3611b816-3169-4f44-a76c-f2e631a1fa37" (UID: "3611b816-3169-4f44-a76c-f2e631a1fa37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:25:45.223515 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.223489    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3611b816-3169-4f44-a76c-f2e631a1fa37-kube-api-access-fvtr8" (OuterVolumeSpecName: "kube-api-access-fvtr8") pod "3611b816-3169-4f44-a76c-f2e631a1fa37" (UID: "3611b816-3169-4f44-a76c-f2e631a1fa37"). InnerVolumeSpecName "kube-api-access-fvtr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:25:45.223604 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.223515    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3611b816-3169-4f44-a76c-f2e631a1fa37-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3611b816-3169-4f44-a76c-f2e631a1fa37" (UID: "3611b816-3169-4f44-a76c-f2e631a1fa37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:25:45.223604 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.223564    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-kube-api-access-5p8s8" (OuterVolumeSpecName: "kube-api-access-5p8s8") pod "fb15a340-07ce-48f1-b9f4-f512a34bd3a0" (UID: "fb15a340-07ce-48f1-b9f4-f512a34bd3a0"). InnerVolumeSpecName "kube-api-access-5p8s8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:25:45.322643 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.322616    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvtr8\" (UniqueName: \"kubernetes.io/projected/3611b816-3169-4f44-a76c-f2e631a1fa37-kube-api-access-fvtr8\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:25:45.322643 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.322638    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:25:45.322795 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.322647    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3611b816-3169-4f44-a76c-f2e631a1fa37-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:25:45.322795 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.322656    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3611b816-3169-4f44-a76c-f2e631a1fa37-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:25:45.322795 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.322663    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p8s8\" (UniqueName: \"kubernetes.io/projected/fb15a340-07ce-48f1-b9f4-f512a34bd3a0-kube-api-access-5p8s8\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:25:45.765631 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.765600    2547 generic.go:356] "Generic (PLEG): container finished" podID="3611b816-3169-4f44-a76c-f2e631a1fa37" containerID="94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488" exitCode=0
Jul 30 21:25:45.765997 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.765637    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"
Jul 30 21:25:45.765997 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.765686    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7" event={"ID":"3611b816-3169-4f44-a76c-f2e631a1fa37","Type":"ContainerDied","Data":"94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488"}
Jul 30 21:25:45.765997 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.765764    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7" event={"ID":"3611b816-3169-4f44-a76c-f2e631a1fa37","Type":"ContainerDied","Data":"14161722c9344bd9a78204dbc192f0c86fe26a97924101717c88dde11776e2b3"}
Jul 30 21:25:45.765997 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.765787    2547 scope.go:117] "RemoveContainer" containerID="94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488"
Jul 30 21:25:45.767133 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.767112    2547 generic.go:356] "Generic (PLEG): container finished" podID="fb15a340-07ce-48f1-b9f4-f512a34bd3a0" containerID="f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b" exitCode=0
Jul 30 21:25:45.767217 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.767156    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"
Jul 30 21:25:45.767217 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.767192    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph" event={"ID":"fb15a340-07ce-48f1-b9f4-f512a34bd3a0","Type":"ContainerDied","Data":"f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b"}
Jul 30 21:25:45.767317 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.767247    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph" event={"ID":"fb15a340-07ce-48f1-b9f4-f512a34bd3a0","Type":"ContainerDied","Data":"30d85409c47e19e40145dac67aa0b259899a4144ffaf60f380c39cb056d7990d"}
Jul 30 21:25:45.768638 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.768614    2547 generic.go:356] "Generic (PLEG): container finished" podID="63c2c76f-66da-49c3-88f9-c474016ebd10" containerID="49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb" exitCode=0
Jul 30 21:25:45.768753 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.768648    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"
Jul 30 21:25:45.768753 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.768725    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7" event={"ID":"63c2c76f-66da-49c3-88f9-c474016ebd10","Type":"ContainerDied","Data":"49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb"}
Jul 30 21:25:45.768845 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.768767    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7" event={"ID":"63c2c76f-66da-49c3-88f9-c474016ebd10","Type":"ContainerDied","Data":"a72dc5047aecd4ed9f62deacf81cb2bd7993a1b95538dd96e3e3abbf4697a415"}
Jul 30 21:25:45.776618 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.776602    2547 scope.go:117] "RemoveContainer" containerID="94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488"
Jul 30 21:25:45.776937 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:25:45.776916    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488\": container with ID starting with 94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488 not found: ID does not exist" containerID="94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488"
Jul 30 21:25:45.777032 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.776943    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488"} err="failed to get container status \"94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488\": rpc error: code = NotFound desc = could not find container \"94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488\": container with ID starting with 94fbcaa16413f2b9f6fb25afc50c065fe1f248dae6295eb13110eec2e0c08488 not found: ID does not exist"
Jul 30 21:25:45.777032 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.776958    2547 scope.go:117] "RemoveContainer" containerID="f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b"
Jul 30 21:25:45.786335 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.786315    2547 scope.go:117] "RemoveContainer" containerID="f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b"
Jul 30 21:25:45.786567 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:25:45.786546    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b\": container with ID starting with f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b not found: ID does not exist" containerID="f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b"
Jul 30 21:25:45.786639 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.786572    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b"} err="failed to get container status \"f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b\": rpc error: code = NotFound desc = could not find container \"f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b\": container with ID starting with f3adf8650461ba1350927254584564df1d3af573cc68721a85a36dca57de768b not found: ID does not exist"
Jul 30 21:25:45.786639 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.786588    2547 scope.go:117] "RemoveContainer" containerID="49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb"
Jul 30 21:25:45.792745 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.792723    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"]
Jul 30 21:25:45.795125 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.795105    2547 scope.go:117] "RemoveContainer" containerID="49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb"
Jul 30 21:25:45.795377 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:25:45.795357    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb\": container with ID starting with 49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb not found: ID does not exist" containerID="49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb"
Jul 30 21:25:45.795446 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.795387    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb"} err="failed to get container status \"49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb\": rpc error: code = NotFound desc = could not find container \"49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb\": container with ID starting with 49dd85d4c2738fd09f80bd90e1fbea9cb377140597eba278b8c6594f29d076fb not found: ID does not exist"
Jul 30 21:25:45.796551 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.796534    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-83307c-7c5979d5db-4ggc7"]
Jul 30 21:25:45.808870 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.808849    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"]
Jul 30 21:25:45.815884 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.815862    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-83307c-777ffbd888-99pp7"]
Jul 30 21:25:45.826635 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.826614    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"]
Jul 30 21:25:45.830720 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:45.830684    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-83307c-589b55fc68-rw5ph"]
Jul 30 21:25:46.072738 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:46.072694    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3611b816-3169-4f44-a76c-f2e631a1fa37" path="/var/lib/kubelet/pods/3611b816-3169-4f44-a76c-f2e631a1fa37/volumes"
Jul 30 21:25:46.073057 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:46.073044    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63c2c76f-66da-49c3-88f9-c474016ebd10" path="/var/lib/kubelet/pods/63c2c76f-66da-49c3-88f9-c474016ebd10/volumes"
Jul 30 21:25:46.073366 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:25:46.073354    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb15a340-07ce-48f1-b9f4-f512a34bd3a0" path="/var/lib/kubelet/pods/fb15a340-07ce-48f1-b9f4-f512a34bd3a0/volumes"
Jul 30 21:26:10.974089 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:10.974055    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"]
Jul 30 21:26:10.974603 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:10.974294    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj" podUID="b6d54db2-5dcf-4221-81f2-48219f3ef989" containerName="maas-api" containerID="cri-o://eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4" gracePeriod=30
Jul 30 21:26:11.026683 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.026653    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"]
Jul 30 21:26:11.026887 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.026858    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h" podUID="10a0caf6-2b40-4d4d-907f-0558af558f5d" containerName="payload-processing" containerID="cri-o://11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376" gracePeriod=30
Jul 30 21:26:11.300992 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.296034    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:26:11.324094 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.324064    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzqhv\" (UniqueName: \"kubernetes.io/projected/b6d54db2-5dcf-4221-81f2-48219f3ef989-kube-api-access-kzqhv\") pod \"b6d54db2-5dcf-4221-81f2-48219f3ef989\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") "
Jul 30 21:26:11.324222 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.324144    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d54db2-5dcf-4221-81f2-48219f3ef989-maas-api-tls\") pod \"b6d54db2-5dcf-4221-81f2-48219f3ef989\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") "
Jul 30 21:26:11.324222 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.324183    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d54db2-5dcf-4221-81f2-48219f3ef989-openshift-service-ca\") pod \"b6d54db2-5dcf-4221-81f2-48219f3ef989\" (UID: \"b6d54db2-5dcf-4221-81f2-48219f3ef989\") "
Jul 30 21:26:11.324870 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.324829    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d54db2-5dcf-4221-81f2-48219f3ef989-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6d54db2-5dcf-4221-81f2-48219f3ef989" (UID: "b6d54db2-5dcf-4221-81f2-48219f3ef989"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:26:11.326733 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.326669    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d54db2-5dcf-4221-81f2-48219f3ef989-kube-api-access-kzqhv" (OuterVolumeSpecName: "kube-api-access-kzqhv") pod "b6d54db2-5dcf-4221-81f2-48219f3ef989" (UID: "b6d54db2-5dcf-4221-81f2-48219f3ef989"). InnerVolumeSpecName "kube-api-access-kzqhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:26:11.326733 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.326685    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6d54db2-5dcf-4221-81f2-48219f3ef989-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6d54db2-5dcf-4221-81f2-48219f3ef989" (UID: "b6d54db2-5dcf-4221-81f2-48219f3ef989"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:26:11.333462 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.333438    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:26:11.425284 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.425254    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgk5d\" (UniqueName: \"kubernetes.io/projected/10a0caf6-2b40-4d4d-907f-0558af558f5d-kube-api-access-bgk5d\") pod \"10a0caf6-2b40-4d4d-907f-0558af558f5d\" (UID: \"10a0caf6-2b40-4d4d-907f-0558af558f5d\") "
Jul 30 21:26:11.425422 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.425365    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a0caf6-2b40-4d4d-907f-0558af558f5d-plugins-config-volume\") pod \"10a0caf6-2b40-4d4d-907f-0558af558f5d\" (UID: \"10a0caf6-2b40-4d4d-907f-0558af558f5d\") "
Jul 30 21:26:11.425671 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.425654    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzqhv\" (UniqueName: \"kubernetes.io/projected/b6d54db2-5dcf-4221-81f2-48219f3ef989-kube-api-access-kzqhv\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:26:11.425774 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.425679    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d54db2-5dcf-4221-81f2-48219f3ef989-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:26:11.425774 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.425690    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a0caf6-2b40-4d4d-907f-0558af558f5d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10a0caf6-2b40-4d4d-907f-0558af558f5d" (UID: "10a0caf6-2b40-4d4d-907f-0558af558f5d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:26:11.425774 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.425693    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d54db2-5dcf-4221-81f2-48219f3ef989-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:26:11.427149 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.427134    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a0caf6-2b40-4d4d-907f-0558af558f5d-kube-api-access-bgk5d" (OuterVolumeSpecName: "kube-api-access-bgk5d") pod "10a0caf6-2b40-4d4d-907f-0558af558f5d" (UID: "10a0caf6-2b40-4d4d-907f-0558af558f5d"). InnerVolumeSpecName "kube-api-access-bgk5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:26:11.526972 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.526906    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgk5d\" (UniqueName: \"kubernetes.io/projected/10a0caf6-2b40-4d4d-907f-0558af558f5d-kube-api-access-bgk5d\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:26:11.526972 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.526933    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a0caf6-2b40-4d4d-907f-0558af558f5d-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:26:11.909557 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.909527    2547 generic.go:356] "Generic (PLEG): container finished" podID="b6d54db2-5dcf-4221-81f2-48219f3ef989" containerID="eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4" exitCode=0
Jul 30 21:26:11.909694 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.909579    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"
Jul 30 21:26:11.909694 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.909618    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj" event={"ID":"b6d54db2-5dcf-4221-81f2-48219f3ef989","Type":"ContainerDied","Data":"eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4"}
Jul 30 21:26:11.909694 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.909672    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj" event={"ID":"b6d54db2-5dcf-4221-81f2-48219f3ef989","Type":"ContainerDied","Data":"9fab13ac9aae070c7026122eb38bb4dafcc49e16a1994ecd14fa1b1f2457d464"}
Jul 30 21:26:11.909899 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.909725    2547 scope.go:117] "RemoveContainer" containerID="eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4"
Jul 30 21:26:11.911143 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.911119    2547 generic.go:356] "Generic (PLEG): container finished" podID="10a0caf6-2b40-4d4d-907f-0558af558f5d" containerID="11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376" exitCode=0
Jul 30 21:26:11.911252 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.911166    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h" event={"ID":"10a0caf6-2b40-4d4d-907f-0558af558f5d","Type":"ContainerDied","Data":"11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376"}
Jul 30 21:26:11.911252 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.911206    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h" event={"ID":"10a0caf6-2b40-4d4d-907f-0558af558f5d","Type":"ContainerDied","Data":"c5200d2cf56e3e920f5f6b6a239193a60bfbe92b50dbaedfde7774eb5fd7497f"}
Jul 30 21:26:11.911337 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.911326    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"
Jul 30 21:26:11.919919 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.919835    2547 scope.go:117] "RemoveContainer" containerID="eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4"
Jul 30 21:26:11.920101 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:26:11.920078    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4\": container with ID starting with eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4 not found: ID does not exist" containerID="eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4"
Jul 30 21:26:11.920182 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.920111    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4"} err="failed to get container status \"eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4\": rpc error: code = NotFound desc = could not find container \"eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4\": container with ID starting with eaecb019ae3500e84e8bc9d2424ef2f5f23f21c2ab5ba383d62fc9545ed051e4 not found: ID does not exist"
Jul 30 21:26:11.920182 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.920133    2547 scope.go:117] "RemoveContainer" containerID="11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376"
Jul 30 21:26:11.927831 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.927814    2547 scope.go:117] "RemoveContainer" containerID="11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376"
Jul 30 21:26:11.928070 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:26:11.928054    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376\": container with ID starting with 11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376 not found: ID does not exist" containerID="11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376"
Jul 30 21:26:11.928116 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.928076    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376"} err="failed to get container status \"11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376\": rpc error: code = NotFound desc = could not find container \"11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376\": container with ID starting with 11e7ba7461d841969ddb3b85dc49824721a689a72066d7fd0d6225f450422376 not found: ID does not exist"
Jul 30 21:26:11.932406 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.932385    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"]
Jul 30 21:26:11.935969 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.935948    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e54afd-77656ff98d-lnqwj"]
Jul 30 21:26:11.946169 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.946150    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"]
Jul 30 21:26:11.950019 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:11.950001    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e54afd-65569fc5b4-42l9h"]
Jul 30 21:26:12.073417 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:12.073395    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10a0caf6-2b40-4d4d-907f-0558af558f5d" path="/var/lib/kubelet/pods/10a0caf6-2b40-4d4d-907f-0558af558f5d/volumes"
Jul 30 21:26:12.073792 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:26:12.073777    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6d54db2-5dcf-4221-81f2-48219f3ef989" path="/var/lib/kubelet/pods/b6d54db2-5dcf-4221-81f2-48219f3ef989/volumes"
Jul 30 21:27:10.503534 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.503500    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"]
Jul 30 21:27:10.504113 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.503793    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v" podUID="c45c325e-03a5-4290-a1f1-b35bacf83425" containerName="maas-api" containerID="cri-o://53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8" gracePeriod=30
Jul 30 21:27:10.769592 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.769569    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:27:10.806738 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.806692    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vd8sd\" (UniqueName: \"kubernetes.io/projected/c45c325e-03a5-4290-a1f1-b35bacf83425-kube-api-access-vd8sd\") pod \"c45c325e-03a5-4290-a1f1-b35bacf83425\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") "
Jul 30 21:27:10.806880 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.806778    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c45c325e-03a5-4290-a1f1-b35bacf83425-maas-api-tls\") pod \"c45c325e-03a5-4290-a1f1-b35bacf83425\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") "
Jul 30 21:27:10.806880 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.806855    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45c325e-03a5-4290-a1f1-b35bacf83425-openshift-service-ca\") pod \"c45c325e-03a5-4290-a1f1-b35bacf83425\" (UID: \"c45c325e-03a5-4290-a1f1-b35bacf83425\") "
Jul 30 21:27:10.807243 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.807215    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c45c325e-03a5-4290-a1f1-b35bacf83425-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c45c325e-03a5-4290-a1f1-b35bacf83425" (UID: "c45c325e-03a5-4290-a1f1-b35bacf83425"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:27:10.809004 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.808978    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c45c325e-03a5-4290-a1f1-b35bacf83425-kube-api-access-vd8sd" (OuterVolumeSpecName: "kube-api-access-vd8sd") pod "c45c325e-03a5-4290-a1f1-b35bacf83425" (UID: "c45c325e-03a5-4290-a1f1-b35bacf83425"). InnerVolumeSpecName "kube-api-access-vd8sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:27:10.809274 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.809250    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c45c325e-03a5-4290-a1f1-b35bacf83425-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c45c325e-03a5-4290-a1f1-b35bacf83425" (UID: "c45c325e-03a5-4290-a1f1-b35bacf83425"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:27:10.907844 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.907817    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45c325e-03a5-4290-a1f1-b35bacf83425-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:10.907844 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.907842    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd8sd\" (UniqueName: \"kubernetes.io/projected/c45c325e-03a5-4290-a1f1-b35bacf83425-kube-api-access-vd8sd\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:10.907983 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:10.907851    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c45c325e-03a5-4290-a1f1-b35bacf83425-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:11.227203 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:11.227169    2547 generic.go:356] "Generic (PLEG): container finished" podID="c45c325e-03a5-4290-a1f1-b35bacf83425" containerID="53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8" exitCode=0
Jul 30 21:27:11.227349 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:11.227208    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"
Jul 30 21:27:11.227349 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:11.227254    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v" event={"ID":"c45c325e-03a5-4290-a1f1-b35bacf83425","Type":"ContainerDied","Data":"53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8"}
Jul 30 21:27:11.227349 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:11.227301    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v" event={"ID":"c45c325e-03a5-4290-a1f1-b35bacf83425","Type":"ContainerDied","Data":"3da423f652740e097f239e5693c33a78fd5629349a7d07ff4daa224a6ae1210d"}
Jul 30 21:27:11.227349 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:11.227323    2547 scope.go:117] "RemoveContainer" containerID="53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8"
Jul 30 21:27:11.236994 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:11.236977    2547 scope.go:117] "RemoveContainer" containerID="53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8"
Jul 30 21:27:11.237263 ip-10-0-134-148 kubenswrapper[2547]: E0730 21:27:11.237240    2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8\": container with ID starting with 53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8 not found: ID does not exist" containerID="53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8"
Jul 30 21:27:11.237321 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:11.237274    2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8"} err="failed to get container status \"53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8\": rpc error: code = NotFound desc = could not find container \"53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8\": container with ID starting with 53b69b3a1c5c9779e935bf4bca2a5d31b6cdbb4e2b1001b0cf01fb2f032adaf8 not found: ID does not exist"
Jul 30 21:27:11.249218 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:11.249198    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"]
Jul 30 21:27:11.253576 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:11.253558    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-296998-fff45cf9c-jv64v"]
Jul 30 21:27:12.074346 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:12.074314    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c45c325e-03a5-4290-a1f1-b35bacf83425" path="/var/lib/kubelet/pods/c45c325e-03a5-4290-a1f1-b35bacf83425/volumes"
Jul 30 21:27:27.959837 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:27.959803    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"]
Jul 30 21:27:27.960328 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:27.960102    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r" podUID="91ce84ee-2915-4d16-977d-e29e65cc7fa3" containerName="maas-api" containerID="cri-o://3d9d2f377a71dfab0b61b4fea729138519cae8700b1246f3e2c0c6616c9a2589" gracePeriod=30
Jul 30 21:27:28.021608 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.021578    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"]
Jul 30 21:27:28.021885 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.021835    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2" podUID="46528b18-2273-4428-8c92-51ed29d88619" containerName="payload-processing" containerID="cri-o://f4a619caf7f7d6259401fffea7ef1f25c8d1d03a93fd834b740147c6bcc89a0c" gracePeriod=30
Jul 30 21:27:28.030345 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.030320    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"]
Jul 30 21:27:28.030958 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.030898    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft" podUID="bdc1d985-0323-4b86-a3e9-26f83ba2b02a" containerName="payload-pre-processing" containerID="cri-o://33c6abb9016aca4acdc36ea08225085a4fd1642656287d4aaaf522222b7ce9f5" gracePeriod=30
Jul 30 21:27:28.314436 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.314410    2547 generic.go:356] "Generic (PLEG): container finished" podID="46528b18-2273-4428-8c92-51ed29d88619" containerID="f4a619caf7f7d6259401fffea7ef1f25c8d1d03a93fd834b740147c6bcc89a0c" exitCode=0
Jul 30 21:27:28.314569 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.314478    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2" event={"ID":"46528b18-2273-4428-8c92-51ed29d88619","Type":"ContainerDied","Data":"f4a619caf7f7d6259401fffea7ef1f25c8d1d03a93fd834b740147c6bcc89a0c"}
Jul 30 21:27:28.316307 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.316285    2547 generic.go:356] "Generic (PLEG): container finished" podID="bdc1d985-0323-4b86-a3e9-26f83ba2b02a" containerID="33c6abb9016aca4acdc36ea08225085a4fd1642656287d4aaaf522222b7ce9f5" exitCode=0
Jul 30 21:27:28.316433 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.316361    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft" event={"ID":"bdc1d985-0323-4b86-a3e9-26f83ba2b02a","Type":"ContainerDied","Data":"33c6abb9016aca4acdc36ea08225085a4fd1642656287d4aaaf522222b7ce9f5"}
Jul 30 21:27:28.318077 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.318056    2547 generic.go:356] "Generic (PLEG): container finished" podID="91ce84ee-2915-4d16-977d-e29e65cc7fa3" containerID="3d9d2f377a71dfab0b61b4fea729138519cae8700b1246f3e2c0c6616c9a2589" exitCode=0
Jul 30 21:27:28.318162 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.318094    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r" event={"ID":"91ce84ee-2915-4d16-977d-e29e65cc7fa3","Type":"ContainerDied","Data":"3d9d2f377a71dfab0b61b4fea729138519cae8700b1246f3e2c0c6616c9a2589"}
Jul 30 21:27:28.338114 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.338084    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:27:28.341238 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.341215    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:27:28.344391 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.344374    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:27:28.440012 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.439983    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ce84ee-2915-4d16-977d-e29e65cc7fa3-maas-api-tls\") pod \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") "
Jul 30 21:27:28.440012 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440016    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ce84ee-2915-4d16-977d-e29e65cc7fa3-openshift-service-ca\") pod \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") "
Jul 30 21:27:28.440239 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440048    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbsfs\" (UniqueName: \"kubernetes.io/projected/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-kube-api-access-qbsfs\") pod \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\" (UID: \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\") "
Jul 30 21:27:28.440239 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440068    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w22rf\" (UniqueName: \"kubernetes.io/projected/46528b18-2273-4428-8c92-51ed29d88619-kube-api-access-w22rf\") pod \"46528b18-2273-4428-8c92-51ed29d88619\" (UID: \"46528b18-2273-4428-8c92-51ed29d88619\") "
Jul 30 21:27:28.440239 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440190    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-plugins-config-volume\") pod \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\" (UID: \"bdc1d985-0323-4b86-a3e9-26f83ba2b02a\") "
Jul 30 21:27:28.440413 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440241    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vs95\" (UniqueName: \"kubernetes.io/projected/91ce84ee-2915-4d16-977d-e29e65cc7fa3-kube-api-access-2vs95\") pod \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\" (UID: \"91ce84ee-2915-4d16-977d-e29e65cc7fa3\") "
Jul 30 21:27:28.440413 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440299    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46528b18-2273-4428-8c92-51ed29d88619-plugins-config-volume\") pod \"46528b18-2273-4428-8c92-51ed29d88619\" (UID: \"46528b18-2273-4428-8c92-51ed29d88619\") "
Jul 30 21:27:28.440522 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440407    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91ce84ee-2915-4d16-977d-e29e65cc7fa3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "91ce84ee-2915-4d16-977d-e29e65cc7fa3" (UID: "91ce84ee-2915-4d16-977d-e29e65cc7fa3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:27:28.440581 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440539    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bdc1d985-0323-4b86-a3e9-26f83ba2b02a" (UID: "bdc1d985-0323-4b86-a3e9-26f83ba2b02a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:27:28.440873 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440845    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46528b18-2273-4428-8c92-51ed29d88619-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "46528b18-2273-4428-8c92-51ed29d88619" (UID: "46528b18-2273-4428-8c92-51ed29d88619"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:27:28.441017 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440970    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ce84ee-2915-4d16-977d-e29e65cc7fa3-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:28.441017 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.440996    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:28.442285 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.442262    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46528b18-2273-4428-8c92-51ed29d88619-kube-api-access-w22rf" (OuterVolumeSpecName: "kube-api-access-w22rf") pod "46528b18-2273-4428-8c92-51ed29d88619" (UID: "46528b18-2273-4428-8c92-51ed29d88619"). InnerVolumeSpecName "kube-api-access-w22rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:27:28.442285 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.442274    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91ce84ee-2915-4d16-977d-e29e65cc7fa3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "91ce84ee-2915-4d16-977d-e29e65cc7fa3" (UID: "91ce84ee-2915-4d16-977d-e29e65cc7fa3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:27:28.442750 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.442729    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-kube-api-access-qbsfs" (OuterVolumeSpecName: "kube-api-access-qbsfs") pod "bdc1d985-0323-4b86-a3e9-26f83ba2b02a" (UID: "bdc1d985-0323-4b86-a3e9-26f83ba2b02a"). InnerVolumeSpecName "kube-api-access-qbsfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:27:28.442750 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.442731    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91ce84ee-2915-4d16-977d-e29e65cc7fa3-kube-api-access-2vs95" (OuterVolumeSpecName: "kube-api-access-2vs95") pod "91ce84ee-2915-4d16-977d-e29e65cc7fa3" (UID: "91ce84ee-2915-4d16-977d-e29e65cc7fa3"). InnerVolumeSpecName "kube-api-access-2vs95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:27:28.542355 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.542302    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ce84ee-2915-4d16-977d-e29e65cc7fa3-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:28.542355 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.542322    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbsfs\" (UniqueName: \"kubernetes.io/projected/bdc1d985-0323-4b86-a3e9-26f83ba2b02a-kube-api-access-qbsfs\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:28.542355 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.542331    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w22rf\" (UniqueName: \"kubernetes.io/projected/46528b18-2273-4428-8c92-51ed29d88619-kube-api-access-w22rf\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:28.542355 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.542339    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vs95\" (UniqueName: \"kubernetes.io/projected/91ce84ee-2915-4d16-977d-e29e65cc7fa3-kube-api-access-2vs95\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:28.542355 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:28.542347    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46528b18-2273-4428-8c92-51ed29d88619-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:29.324148 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.324122    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"
Jul 30 21:27:29.324545 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.324141    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft" event={"ID":"bdc1d985-0323-4b86-a3e9-26f83ba2b02a","Type":"ContainerDied","Data":"67f11c8ae5d02180649e60242f0e61b2d99533a563217b15366ae53769ff94b5"}
Jul 30 21:27:29.324545 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.324223    2547 scope.go:117] "RemoveContainer" containerID="33c6abb9016aca4acdc36ea08225085a4fd1642656287d4aaaf522222b7ce9f5"
Jul 30 21:27:29.325474 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.325449    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"
Jul 30 21:27:29.325548 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.325461    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r" event={"ID":"91ce84ee-2915-4d16-977d-e29e65cc7fa3","Type":"ContainerDied","Data":"a4803435aebb5b08598ea7975d65e9e32cbc5d511d2db0cd4d152ff1c071adf9"}
Jul 30 21:27:29.326851 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.326823    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"
Jul 30 21:27:29.326956 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.326870    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2" event={"ID":"46528b18-2273-4428-8c92-51ed29d88619","Type":"ContainerDied","Data":"c30b8b8f2e2061e17c7ad715326815d4d2b13e25ece36b0859366621eccab5ff"}
Jul 30 21:27:29.336090 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.336074    2547 scope.go:117] "RemoveContainer" containerID="3d9d2f377a71dfab0b61b4fea729138519cae8700b1246f3e2c0c6616c9a2589"
Jul 30 21:27:29.343978 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.343961    2547 scope.go:117] "RemoveContainer" containerID="f4a619caf7f7d6259401fffea7ef1f25c8d1d03a93fd834b740147c6bcc89a0c"
Jul 30 21:27:29.352602 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.352583    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"]
Jul 30 21:27:29.356579 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.356559    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-155c1d-68cffd4b84-dqm9r"]
Jul 30 21:27:29.367311 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.367293    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"]
Jul 30 21:27:29.371112 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.371092    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-155c1d-65f4db4596-jkdft"]
Jul 30 21:27:29.381595 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.381570    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"]
Jul 30 21:27:29.385661 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:29.385638    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-155c1d-8fc67c8b-vgmq2"]
Jul 30 21:27:30.072720 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:30.072680    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46528b18-2273-4428-8c92-51ed29d88619" path="/var/lib/kubelet/pods/46528b18-2273-4428-8c92-51ed29d88619/volumes"
Jul 30 21:27:30.073083 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:30.073067    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91ce84ee-2915-4d16-977d-e29e65cc7fa3" path="/var/lib/kubelet/pods/91ce84ee-2915-4d16-977d-e29e65cc7fa3/volumes"
Jul 30 21:27:30.073368 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:30.073356    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdc1d985-0323-4b86-a3e9-26f83ba2b02a" path="/var/lib/kubelet/pods/bdc1d985-0323-4b86-a3e9-26f83ba2b02a/volumes"
Jul 30 21:27:35.350030 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.349997    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"]
Jul 30 21:27:35.350428 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350376    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdc1d985-0323-4b86-a3e9-26f83ba2b02a" containerName="payload-pre-processing"
Jul 30 21:27:35.350428 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350386    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdc1d985-0323-4b86-a3e9-26f83ba2b02a" containerName="payload-pre-processing"
Jul 30 21:27:35.350428 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350392    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91ce84ee-2915-4d16-977d-e29e65cc7fa3" containerName="maas-api"
Jul 30 21:27:35.350428 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350399    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="91ce84ee-2915-4d16-977d-e29e65cc7fa3" containerName="maas-api"
Jul 30 21:27:35.350428 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350413    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c45c325e-03a5-4290-a1f1-b35bacf83425" containerName="maas-api"
Jul 30 21:27:35.350428 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350419    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="c45c325e-03a5-4290-a1f1-b35bacf83425" containerName="maas-api"
Jul 30 21:27:35.350428 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350431    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10a0caf6-2b40-4d4d-907f-0558af558f5d" containerName="payload-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350437    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="10a0caf6-2b40-4d4d-907f-0558af558f5d" containerName="payload-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350443    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63c2c76f-66da-49c3-88f9-c474016ebd10" containerName="payload-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350448    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="63c2c76f-66da-49c3-88f9-c474016ebd10" containerName="payload-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350460    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d54db2-5dcf-4221-81f2-48219f3ef989" containerName="maas-api"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350465    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d54db2-5dcf-4221-81f2-48219f3ef989" containerName="maas-api"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350470    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb15a340-07ce-48f1-b9f4-f512a34bd3a0" containerName="payload-pre-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350474    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb15a340-07ce-48f1-b9f4-f512a34bd3a0" containerName="payload-pre-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350482    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3611b816-3169-4f44-a76c-f2e631a1fa37" containerName="maas-api"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350487    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="3611b816-3169-4f44-a76c-f2e631a1fa37" containerName="maas-api"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350493    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46528b18-2273-4428-8c92-51ed29d88619" containerName="payload-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350498    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="46528b18-2273-4428-8c92-51ed29d88619" containerName="payload-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350553    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb15a340-07ce-48f1-b9f4-f512a34bd3a0" containerName="payload-pre-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350561    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="63c2c76f-66da-49c3-88f9-c474016ebd10" containerName="payload-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350567    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="46528b18-2273-4428-8c92-51ed29d88619" containerName="payload-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350574    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6d54db2-5dcf-4221-81f2-48219f3ef989" containerName="maas-api"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350579    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="3611b816-3169-4f44-a76c-f2e631a1fa37" containerName="maas-api"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350585    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="91ce84ee-2915-4d16-977d-e29e65cc7fa3" containerName="maas-api"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350589    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="c45c325e-03a5-4290-a1f1-b35bacf83425" containerName="maas-api"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350594    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdc1d985-0323-4b86-a3e9-26f83ba2b02a" containerName="payload-pre-processing"
Jul 30 21:27:35.350652 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.350599    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="10a0caf6-2b40-4d4d-907f-0558af558f5d" containerName="payload-processing"
Jul 30 21:27:35.354972 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.354953    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"
Jul 30 21:27:35.357449 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.357424    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zm2hs\""
Jul 30 21:27:35.361791 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.361770    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"]
Jul 30 21:27:35.395155 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.395130    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417552f-4e7c-4914-8dc6-2bb2e75f1622-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh\" (UID: \"c417552f-4e7c-4914-8dc6-2bb2e75f1622\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"
Jul 30 21:27:35.496375 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.496348    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417552f-4e7c-4914-8dc6-2bb2e75f1622-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh\" (UID: \"c417552f-4e7c-4914-8dc6-2bb2e75f1622\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"
Jul 30 21:27:35.496946 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.496924    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417552f-4e7c-4914-8dc6-2bb2e75f1622-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh\" (UID: \"c417552f-4e7c-4914-8dc6-2bb2e75f1622\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"
Jul 30 21:27:35.667093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.667037    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"
Jul 30 21:27:35.806714 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:35.806670    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"]
Jul 30 21:27:35.808143 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:27:35.808113    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc417552f_4e7c_4914_8dc6_2bb2e75f1622.slice/crio-91b92ecef225da1e06dda1598418e866315bfc0f08012e835694b25bdf193680 WatchSource:0}: Error finding container 91b92ecef225da1e06dda1598418e866315bfc0f08012e835694b25bdf193680: Status 404 returned error can't find the container with id 91b92ecef225da1e06dda1598418e866315bfc0f08012e835694b25bdf193680
Jul 30 21:27:36.370302 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:36.370271    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh" event={"ID":"c417552f-4e7c-4914-8dc6-2bb2e75f1622","Type":"ContainerStarted","Data":"f9c8cdd20734efce21663e006a4383c6d4d573eba8fa56b68a15490e819b1066"}
Jul 30 21:27:36.370691 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:36.370321    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh" event={"ID":"c417552f-4e7c-4914-8dc6-2bb2e75f1622","Type":"ContainerStarted","Data":"91b92ecef225da1e06dda1598418e866315bfc0f08012e835694b25bdf193680"}
Jul 30 21:27:36.386781 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:36.386737    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh" podStartSLOduration=1.386723537 podStartE2EDuration="1.386723537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 21:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:27:36.385107979 +0000 UTC m=+3734.846066792" watchObservedRunningTime="2026-07-30 21:27:36.386723537 +0000 UTC m=+3734.847682365"
Jul 30 21:27:37.376436 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:37.376405    2547 generic.go:356] "Generic (PLEG): container finished" podID="c417552f-4e7c-4914-8dc6-2bb2e75f1622" containerID="f9c8cdd20734efce21663e006a4383c6d4d573eba8fa56b68a15490e819b1066" exitCode=0
Jul 30 21:27:37.376830 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:37.376494    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh" event={"ID":"c417552f-4e7c-4914-8dc6-2bb2e75f1622","Type":"ContainerDied","Data":"f9c8cdd20734efce21663e006a4383c6d4d573eba8fa56b68a15490e819b1066"}
Jul 30 21:27:38.502249 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:38.502224    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"
Jul 30 21:27:38.621931 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:38.621909    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417552f-4e7c-4914-8dc6-2bb2e75f1622-maas-api-service-ca\") pod \"c417552f-4e7c-4914-8dc6-2bb2e75f1622\" (UID: \"c417552f-4e7c-4914-8dc6-2bb2e75f1622\") "
Jul 30 21:27:38.622226 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:38.622205    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c417552f-4e7c-4914-8dc6-2bb2e75f1622-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c417552f-4e7c-4914-8dc6-2bb2e75f1622" (UID: "c417552f-4e7c-4914-8dc6-2bb2e75f1622"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:27:38.722883 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:38.722827    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c417552f-4e7c-4914-8dc6-2bb2e75f1622-maas-api-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:39.388341 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:39.388318    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"
Jul 30 21:27:39.388515 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:39.388311    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh" event={"ID":"c417552f-4e7c-4914-8dc6-2bb2e75f1622","Type":"ContainerDied","Data":"91b92ecef225da1e06dda1598418e866315bfc0f08012e835694b25bdf193680"}
Jul 30 21:27:39.388515 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:39.388441    2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91b92ecef225da1e06dda1598418e866315bfc0f08012e835694b25bdf193680"
Jul 30 21:27:40.117157 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.117127    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"]
Jul 30 21:27:40.117584 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.117298    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2" podUID="57f30a57-067a-4dfe-9142-b71b137a53db" containerName="maas-api" containerID="cri-o://2dec810def139848cf185f16e0f0955d385379db50111a614ae92bec218acbda" gracePeriod=30
Jul 30 21:27:40.175149 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.175111    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"]
Jul 30 21:27:40.175351 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.175303    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k" podUID="afad7662-a865-4a3c-813b-da51986a06d7" containerName="payload-processing" containerID="cri-o://a2e59a30a2e4ef573a4089c059fe60329ffa869d64f313c0b0b3cec50af89ba1" gracePeriod=30
Jul 30 21:27:40.181196 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.181167    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"]
Jul 30 21:27:40.181430 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.181377    2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2" podUID="e3c07009-f54d-40fb-8607-0adb22f99e1c" containerName="payload-pre-processing" containerID="cri-o://a9c589ae08c227cc8c530e2b01f1979de18304c61e772e929be28bb8dcf9aa24" gracePeriod=30
Jul 30 21:27:40.396990 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.396962    2547 generic.go:356] "Generic (PLEG): container finished" podID="57f30a57-067a-4dfe-9142-b71b137a53db" containerID="2dec810def139848cf185f16e0f0955d385379db50111a614ae92bec218acbda" exitCode=0
Jul 30 21:27:40.397111 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.397089    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2" event={"ID":"57f30a57-067a-4dfe-9142-b71b137a53db","Type":"ContainerDied","Data":"2dec810def139848cf185f16e0f0955d385379db50111a614ae92bec218acbda"}
Jul 30 21:27:40.398947 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.398917    2547 generic.go:356] "Generic (PLEG): container finished" podID="e3c07009-f54d-40fb-8607-0adb22f99e1c" containerID="a9c589ae08c227cc8c530e2b01f1979de18304c61e772e929be28bb8dcf9aa24" exitCode=0
Jul 30 21:27:40.399116 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.399076    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2" event={"ID":"e3c07009-f54d-40fb-8607-0adb22f99e1c","Type":"ContainerDied","Data":"a9c589ae08c227cc8c530e2b01f1979de18304c61e772e929be28bb8dcf9aa24"}
Jul 30 21:27:40.401190 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.401064    2547 generic.go:356] "Generic (PLEG): container finished" podID="afad7662-a865-4a3c-813b-da51986a06d7" containerID="a2e59a30a2e4ef573a4089c059fe60329ffa869d64f313c0b0b3cec50af89ba1" exitCode=0
Jul 30 21:27:40.401190 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.401136    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k" event={"ID":"afad7662-a865-4a3c-813b-da51986a06d7","Type":"ContainerDied","Data":"a2e59a30a2e4ef573a4089c059fe60329ffa869d64f313c0b0b3cec50af89ba1"}
Jul 30 21:27:40.420059 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.420015    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"]
Jul 30 21:27:40.423166 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.423141    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-01bc1b-6xcnh"]
Jul 30 21:27:40.439386 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.439360    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:27:40.458443 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.458425    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:27:40.481629 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.481608    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:27:40.536785 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.536756    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57f30a57-067a-4dfe-9142-b71b137a53db-openshift-service-ca\") pod \"57f30a57-067a-4dfe-9142-b71b137a53db\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") "
Jul 30 21:27:40.536954 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.536801    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcn4c\" (UniqueName: \"kubernetes.io/projected/57f30a57-067a-4dfe-9142-b71b137a53db-kube-api-access-pcn4c\") pod \"57f30a57-067a-4dfe-9142-b71b137a53db\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") "
Jul 30 21:27:40.536954 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.536871    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afad7662-a865-4a3c-813b-da51986a06d7-plugins-config-volume\") pod \"afad7662-a865-4a3c-813b-da51986a06d7\" (UID: \"afad7662-a865-4a3c-813b-da51986a06d7\") "
Jul 30 21:27:40.536954 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.536935    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2kd7\" (UniqueName: \"kubernetes.io/projected/e3c07009-f54d-40fb-8607-0adb22f99e1c-kube-api-access-b2kd7\") pod \"e3c07009-f54d-40fb-8607-0adb22f99e1c\" (UID: \"e3c07009-f54d-40fb-8607-0adb22f99e1c\") "
Jul 30 21:27:40.537122 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.536966    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57f30a57-067a-4dfe-9142-b71b137a53db-maas-api-tls\") pod \"57f30a57-067a-4dfe-9142-b71b137a53db\" (UID: \"57f30a57-067a-4dfe-9142-b71b137a53db\") "
Jul 30 21:27:40.537122 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.537020    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ktsd\" (UniqueName: \"kubernetes.io/projected/afad7662-a865-4a3c-813b-da51986a06d7-kube-api-access-6ktsd\") pod \"afad7662-a865-4a3c-813b-da51986a06d7\" (UID: \"afad7662-a865-4a3c-813b-da51986a06d7\") "
Jul 30 21:27:40.537122 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.537054    2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3c07009-f54d-40fb-8607-0adb22f99e1c-plugins-config-volume\") pod \"e3c07009-f54d-40fb-8607-0adb22f99e1c\" (UID: \"e3c07009-f54d-40fb-8607-0adb22f99e1c\") "
Jul 30 21:27:40.537280 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.537145    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57f30a57-067a-4dfe-9142-b71b137a53db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57f30a57-067a-4dfe-9142-b71b137a53db" (UID: "57f30a57-067a-4dfe-9142-b71b137a53db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:27:40.537945 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.537320    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afad7662-a865-4a3c-813b-da51986a06d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "afad7662-a865-4a3c-813b-da51986a06d7" (UID: "afad7662-a865-4a3c-813b-da51986a06d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:27:40.537945 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.537354    2547 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57f30a57-067a-4dfe-9142-b71b137a53db-openshift-service-ca\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:40.537945 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.537906    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c07009-f54d-40fb-8607-0adb22f99e1c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3c07009-f54d-40fb-8607-0adb22f99e1c" (UID: "e3c07009-f54d-40fb-8607-0adb22f99e1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 21:27:40.539166 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.539131    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3c07009-f54d-40fb-8607-0adb22f99e1c-kube-api-access-b2kd7" (OuterVolumeSpecName: "kube-api-access-b2kd7") pod "e3c07009-f54d-40fb-8607-0adb22f99e1c" (UID: "e3c07009-f54d-40fb-8607-0adb22f99e1c"). InnerVolumeSpecName "kube-api-access-b2kd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:27:40.539273 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.539229    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f30a57-067a-4dfe-9142-b71b137a53db-kube-api-access-pcn4c" (OuterVolumeSpecName: "kube-api-access-pcn4c") pod "57f30a57-067a-4dfe-9142-b71b137a53db" (UID: "57f30a57-067a-4dfe-9142-b71b137a53db"). InnerVolumeSpecName "kube-api-access-pcn4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:27:40.539436 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.539417    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afad7662-a865-4a3c-813b-da51986a06d7-kube-api-access-6ktsd" (OuterVolumeSpecName: "kube-api-access-6ktsd") pod "afad7662-a865-4a3c-813b-da51986a06d7" (UID: "afad7662-a865-4a3c-813b-da51986a06d7"). InnerVolumeSpecName "kube-api-access-6ktsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 21:27:40.539604 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.539582    2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f30a57-067a-4dfe-9142-b71b137a53db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57f30a57-067a-4dfe-9142-b71b137a53db" (UID: "57f30a57-067a-4dfe-9142-b71b137a53db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 21:27:40.637813 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.637734    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2kd7\" (UniqueName: \"kubernetes.io/projected/e3c07009-f54d-40fb-8607-0adb22f99e1c-kube-api-access-b2kd7\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:40.637813 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.637763    2547 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57f30a57-067a-4dfe-9142-b71b137a53db-maas-api-tls\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:40.637813 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.637772    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ktsd\" (UniqueName: \"kubernetes.io/projected/afad7662-a865-4a3c-813b-da51986a06d7-kube-api-access-6ktsd\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:40.637813 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.637781    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3c07009-f54d-40fb-8607-0adb22f99e1c-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:40.637813 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.637789    2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcn4c\" (UniqueName: \"kubernetes.io/projected/57f30a57-067a-4dfe-9142-b71b137a53db-kube-api-access-pcn4c\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:40.637813 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:40.637797    2547 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afad7662-a865-4a3c-813b-da51986a06d7-plugins-config-volume\") on node \"ip-10-0-134-148.ec2.internal\" DevicePath \"\""
Jul 30 21:27:41.407526 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.407447    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"
Jul 30 21:27:41.407526 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.407466    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2" event={"ID":"57f30a57-067a-4dfe-9142-b71b137a53db","Type":"ContainerDied","Data":"1b5496cd2882ab461b34669ede199fcebaa8d9e1f682629706bd37e4eb32f95f"}
Jul 30 21:27:41.408077 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.407543    2547 scope.go:117] "RemoveContainer" containerID="2dec810def139848cf185f16e0f0955d385379db50111a614ae92bec218acbda"
Jul 30 21:27:41.408964 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.408837    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"
Jul 30 21:27:41.408964 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.408875    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2" event={"ID":"e3c07009-f54d-40fb-8607-0adb22f99e1c","Type":"ContainerDied","Data":"129a39df58498c6e9f6fe022d8afcd874fffc17ff553fe20fdc282ea60f9605b"}
Jul 30 21:27:41.410377 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.410354    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k" event={"ID":"afad7662-a865-4a3c-813b-da51986a06d7","Type":"ContainerDied","Data":"7ebb8e573213c68dabdb64f8b6b2224403f34f4483a646f84f0eeb997c0fbb80"}
Jul 30 21:27:41.410505 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.410425    2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"
Jul 30 21:27:41.418336 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.418264    2547 scope.go:117] "RemoveContainer" containerID="a9c589ae08c227cc8c530e2b01f1979de18304c61e772e929be28bb8dcf9aa24"
Jul 30 21:27:41.426968 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.426952    2547 scope.go:117] "RemoveContainer" containerID="a2e59a30a2e4ef573a4089c059fe60329ffa869d64f313c0b0b3cec50af89ba1"
Jul 30 21:27:41.435305 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.435286    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"]
Jul 30 21:27:41.439289 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.439270    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-01bc1b-6cbcb6f787-2ztj2"]
Jul 30 21:27:41.449523 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.449505    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"]
Jul 30 21:27:41.459166 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.459147    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-01bc1b-88dcd4b68-t64c2"]
Jul 30 21:27:41.469850 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.469831    2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"]
Jul 30 21:27:41.473159 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:41.473139    2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-01bc1b-74f7cf7666-wnr7k"]
Jul 30 21:27:42.071928 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:42.071900    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57f30a57-067a-4dfe-9142-b71b137a53db" path="/var/lib/kubelet/pods/57f30a57-067a-4dfe-9142-b71b137a53db/volumes"
Jul 30 21:27:42.072232 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:42.072219    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afad7662-a865-4a3c-813b-da51986a06d7" path="/var/lib/kubelet/pods/afad7662-a865-4a3c-813b-da51986a06d7/volumes"
Jul 30 21:27:42.072493 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:42.072481    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c417552f-4e7c-4914-8dc6-2bb2e75f1622" path="/var/lib/kubelet/pods/c417552f-4e7c-4914-8dc6-2bb2e75f1622/volumes"
Jul 30 21:27:42.072790 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:42.072778    2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3c07009-f54d-40fb-8607-0adb22f99e1c" path="/var/lib/kubelet/pods/e3c07009-f54d-40fb-8607-0adb22f99e1c/volumes"
Jul 30 21:27:47.785322 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:47.785290    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-db9bdb8cf-fhblv_e5cf6ad1-708e-4c06-a744-5d6c69f1893d/authorino/0.log"
Jul 30 21:27:53.679377 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:53.679350    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dldl8_63b4f0a2-c8a6-40a6-b00f-cd981d354b26/server/0.log"
Jul 30 21:27:53.798688 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:53.798659    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8nzgh_fd289d15-b395-4a6b-94e2-36500578a89e/manager/0.log"
Jul 30 21:27:54.035383 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:54.035332    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fd799f58b-skt94_71fe5fe6-416c-4479-8fa0-be872a6157e5/copy-manifests/0.log"
Jul 30 21:27:54.041929 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:54.041914    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fd799f58b-skt94_71fe5fe6-416c-4479-8fa0-be872a6157e5/manager/0.log"
Jul 30 21:27:54.521076 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:54.521043    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-697c5f7f45-bdgw5_89fed71f-29a0-4055-a6f2-0964f289bbe1/maas-api/0.log"
Jul 30 21:27:54.633294 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:54.633269    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757405-ghlmg_4e997265-01b6-4575-999c-f16ec66d7966/cleanup/0.log"
Jul 30 21:27:54.973017 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:54.973000    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-mhmtl_a7837f60-07d7-48e0-9a58-193f428bb5b3/postgres/0.log"
Jul 30 21:27:55.763134 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:55.763115    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-db9bdb8cf-fhblv_e5cf6ad1-708e-4c06-a744-5d6c69f1893d/authorino/0.log"
Jul 30 21:27:55.884119 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:55.884097    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-llkvp_782e133f-136b-456b-af50-3724115aeda7/manager/0.log"
Jul 30 21:27:55.995658 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:55.995635    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-ftk76_6b5ea85f-7ef7-445e-89af-0c2febe43f59/manager/0.log"
Jul 30 21:27:56.223437 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:56.223420    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-lv67d_bce408fb-a78e-4138-b776-855e915eadb8/manager/0.log"
Jul 30 21:27:56.958111 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:56.958089    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ccnsn_b944a48c-5904-4e31-b715-765956c18303/discovery/0.log"
Jul 30 21:27:57.073827 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:57.073803    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66684cbb98-8mqzf_a1996729-7f1a-45fe-8544-fc47129da3c2/kube-auth-proxy/0.log"
Jul 30 21:27:57.440960 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:57.440941    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-56rxl_74572307-3be4-4d10-a667-a7ccfbf839ba/payload-pre-processing/0.log"
Jul 30 21:27:57.677249 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:57.677231    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68cfb99b6d-9cgw6_b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3/router/0.log"
Jul 30 21:27:58.283631 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:58.283606    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-6l46d_3f113f75-182b-4e0d-a80b-aa068d2e222e/main/0.log"
Jul 30 21:27:58.524967 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:27:58.524941    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-k9dmh_bd0c2c7a-902c-4b86-a157-b4ca0ab7a665/main/0.log"
Jul 30 21:28:07.845517 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:07.845490    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9ttmg_643b0bcf-2222-46a0-b8ba-5d556c74e5a3/global-pull-secret-syncer/0.log"
Jul 30 21:28:07.977580 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:07.977551    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qbvzb_9ee03f66-de6a-40c8-a2c0-534ac1754eb2/konnectivity-agent/0.log"
Jul 30 21:28:08.042754 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:08.042733    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-148.ec2.internal_858f9e0ac50469cf471404be973e16f5/haproxy/0.log"
Jul 30 21:28:12.584175 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:12.584141    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-db9bdb8cf-fhblv_e5cf6ad1-708e-4c06-a744-5d6c69f1893d/authorino/0.log"
Jul 30 21:28:12.618718 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:12.618681    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-llkvp_782e133f-136b-456b-af50-3724115aeda7/manager/0.log"
Jul 30 21:28:12.650168 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:12.650113    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-ftk76_6b5ea85f-7ef7-445e-89af-0c2febe43f59/manager/0.log"
Jul 30 21:28:12.765397 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:12.765372    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-lv67d_bce408fb-a78e-4138-b776-855e915eadb8/manager/0.log"
Jul 30 21:28:14.288022 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.287993    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_905745e6-0eb4-4fb0-bba1-c638349052c4/alertmanager/0.log"
Jul 30 21:28:14.312604 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.312583    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_905745e6-0eb4-4fb0-bba1-c638349052c4/config-reloader/0.log"
Jul 30 21:28:14.336261 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.336235    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_905745e6-0eb4-4fb0-bba1-c638349052c4/kube-rbac-proxy-web/0.log"
Jul 30 21:28:14.360942 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.360924    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_905745e6-0eb4-4fb0-bba1-c638349052c4/kube-rbac-proxy/0.log"
Jul 30 21:28:14.382667 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.382645    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_905745e6-0eb4-4fb0-bba1-c638349052c4/kube-rbac-proxy-metric/0.log"
Jul 30 21:28:14.405737 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.405693    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_905745e6-0eb4-4fb0-bba1-c638349052c4/prom-label-proxy/0.log"
Jul 30 21:28:14.427250 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.427232    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_905745e6-0eb4-4fb0-bba1-c638349052c4/init-config-reloader/0.log"
Jul 30 21:28:14.507887 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.507868    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-6mp2w_f4b35782-fbda-4b14-b357-aeef3e1a6265/kube-state-metrics/0.log"
Jul 30 21:28:14.527037 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.527018    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-6mp2w_f4b35782-fbda-4b14-b357-aeef3e1a6265/kube-rbac-proxy-main/0.log"
Jul 30 21:28:14.548569 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.548514    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-6mp2w_f4b35782-fbda-4b14-b357-aeef3e1a6265/kube-rbac-proxy-self/0.log"
Jul 30 21:28:14.574007 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.573981    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-65f7ff9d9b-bghbb_a4e1305d-550b-425a-a804-b73dad9fdd54/metrics-server/0.log"
Jul 30 21:28:14.646091 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.646070    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6x9d6_e6bc63d6-58e6-481c-8900-bd67d7942342/node-exporter/0.log"
Jul 30 21:28:14.667478 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.667456    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6x9d6_e6bc63d6-58e6-481c-8900-bd67d7942342/kube-rbac-proxy/0.log"
Jul 30 21:28:14.692133 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:14.692118    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6x9d6_e6bc63d6-58e6-481c-8900-bd67d7942342/init-textfile/0.log"
Jul 30 21:28:17.574412 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:17.574381    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/2.log"
Jul 30 21:28:17.578600 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:17.578580    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-7xmcm_8fe3690a-68d4-4e16-9871-c258935edb0e/console-operator/3.log"
Jul 30 21:28:18.113586 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.113555    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6554b88db9-7bgfv_c144dbdf-cb96-4569-a126-ecac59770f24/console/0.log"
Jul 30 21:28:18.152351 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.152333    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-gdngc_cba71dcb-757a-4991-8831-6899b4e8876a/download-server/0.log"
Jul 30 21:28:18.531965 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.531905    2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"]
Jul 30 21:28:18.532247 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532235    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3c07009-f54d-40fb-8607-0adb22f99e1c" containerName="payload-pre-processing"
Jul 30 21:28:18.532307 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532248    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3c07009-f54d-40fb-8607-0adb22f99e1c" containerName="payload-pre-processing"
Jul 30 21:28:18.532307 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532269    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c417552f-4e7c-4914-8dc6-2bb2e75f1622" containerName="revoke-keys"
Jul 30 21:28:18.532307 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532275    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="c417552f-4e7c-4914-8dc6-2bb2e75f1622" containerName="revoke-keys"
Jul 30 21:28:18.532307 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532285    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f30a57-067a-4dfe-9142-b71b137a53db" containerName="maas-api"
Jul 30 21:28:18.532307 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532290    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f30a57-067a-4dfe-9142-b71b137a53db" containerName="maas-api"
Jul 30 21:28:18.532307 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532299    2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afad7662-a865-4a3c-813b-da51986a06d7" containerName="payload-processing"
Jul 30 21:28:18.532307 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532304    2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="afad7662-a865-4a3c-813b-da51986a06d7" containerName="payload-processing"
Jul 30 21:28:18.532672 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532416    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="afad7662-a865-4a3c-813b-da51986a06d7" containerName="payload-processing"
Jul 30 21:28:18.532672 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532430    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3c07009-f54d-40fb-8607-0adb22f99e1c" containerName="payload-pre-processing"
Jul 30 21:28:18.532672 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532444    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f30a57-067a-4dfe-9142-b71b137a53db" containerName="maas-api"
Jul 30 21:28:18.532672 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.532450    2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="c417552f-4e7c-4914-8dc6-2bb2e75f1622" containerName="revoke-keys"
Jul 30 21:28:18.535776 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.535760    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.538494 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.538465    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9kzmr\"/\"openshift-service-ca.crt\""
Jul 30 21:28:18.538584 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.538504    2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9kzmr\"/\"kube-root-ca.crt\""
Jul 30 21:28:18.539781 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.539755    2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9kzmr\"/\"default-dockercfg-xgkwf\""
Jul 30 21:28:18.545149 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.545129    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"]
Jul 30 21:28:18.718258 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.718230    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-lib-modules\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.718601 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.718264    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk9fs\" (UniqueName: \"kubernetes.io/projected/cc2afddb-6374-414a-b679-08f5751f60d5-kube-api-access-zk9fs\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.718601 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.718348    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-podres\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.718601 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.718408    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-proc\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.718601 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.718444    2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-sys\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.818967 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.818911    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-proc\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.818967 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.818945    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-sys\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.819093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.818972    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-lib-modules\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.819093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.818989    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk9fs\" (UniqueName: \"kubernetes.io/projected/cc2afddb-6374-414a-b679-08f5751f60d5-kube-api-access-zk9fs\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.819093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.819031    2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-podres\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.819093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.819038    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-proc\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.819093 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.819038    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-sys\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.819250 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.819113    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-lib-modules\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.819250 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.819130    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cc2afddb-6374-414a-b679-08f5751f60d5-podres\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.827199 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.827173    2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk9fs\" (UniqueName: \"kubernetes.io/projected/cc2afddb-6374-414a-b679-08f5751f60d5-kube-api-access-zk9fs\") pod \"perf-node-gather-daemonset-j8mx8\" (UID: \"cc2afddb-6374-414a-b679-08f5751f60d5\") " pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.847285 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.847263    2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:18.980759 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:18.980731    2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"]
Jul 30 21:28:18.982583 ip-10-0-134-148 kubenswrapper[2547]: W0730 21:28:18.982549    2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcc2afddb_6374_414a_b679_08f5751f60d5.slice/crio-de279826b1dd133a52b7a237e2ca07d88a66c8e280f646c58b2e1c297944fdc4 WatchSource:0}: Error finding container de279826b1dd133a52b7a237e2ca07d88a66c8e280f646c58b2e1c297944fdc4: Status 404 returned error can't find the container with id de279826b1dd133a52b7a237e2ca07d88a66c8e280f646c58b2e1c297944fdc4
Jul 30 21:28:19.598194 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:19.598164    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8" event={"ID":"cc2afddb-6374-414a-b679-08f5751f60d5","Type":"ContainerStarted","Data":"bf582cdc69f9982c3cb94b31bdb08b2c95e1de5ea1cefb3c830294e04edb5fb4"}
Jul 30 21:28:19.598342 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:19.598211    2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8" event={"ID":"cc2afddb-6374-414a-b679-08f5751f60d5","Type":"ContainerStarted","Data":"de279826b1dd133a52b7a237e2ca07d88a66c8e280f646c58b2e1c297944fdc4"}
Jul 30 21:28:19.598342 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:19.598307    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:19.615270 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:19.615228    2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8" podStartSLOduration=1.6152152929999999 podStartE2EDuration="1.615215293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 21:28:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 21:28:19.614337906 +0000 UTC m=+3778.075296718" watchObservedRunningTime="2026-07-30 21:28:19.615215293 +0000 UTC m=+3778.076174140"
Jul 30 21:28:20.026720 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:20.026650    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tjtsk_4bfaea5d-a4b2-42a0-889d-29b157f75146/dns/0.log"
Jul 30 21:28:20.049270 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:20.049250    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tjtsk_4bfaea5d-a4b2-42a0-889d-29b157f75146/kube-rbac-proxy/0.log"
Jul 30 21:28:20.141136 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:20.141120    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8x4qs_029812bb-2174-49d1-9bc2-9d576b3c2c80/dns-node-resolver/0.log"
Jul 30 21:28:20.710342 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:20.710326    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8kxwd_17492ecc-9882-4680-9df3-5ef550df1e0d/node-ca/0.log"
Jul 30 21:28:21.846129 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:21.846101    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ccnsn_b944a48c-5904-4e31-b715-765956c18303/discovery/0.log"
Jul 30 21:28:21.866979 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:21.866953    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66684cbb98-8mqzf_a1996729-7f1a-45fe-8544-fc47129da3c2/kube-auth-proxy/0.log"
Jul 30 21:28:21.963942 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:21.963918    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-56rxl_74572307-3be4-4d10-a667-a7ccfbf839ba/payload-pre-processing/0.log"
Jul 30 21:28:22.019000 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:22.018982    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68cfb99b6d-9cgw6_b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3/router/0.log"
Jul 30 21:28:22.803273 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:22.803215    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-48ck8_ae70ec57-f212-4922-8ea3-c2704e36af32/serve-healthcheck-canary/0.log"
Jul 30 21:28:23.427755 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:23.427734    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m2hjb_4ce2fcd5-97a7-4799-9dcb-5806750659e5/kube-rbac-proxy/0.log"
Jul 30 21:28:23.452301 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:23.452274    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m2hjb_4ce2fcd5-97a7-4799-9dcb-5806750659e5/exporter/0.log"
Jul 30 21:28:23.473612 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:23.473591    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m2hjb_4ce2fcd5-97a7-4799-9dcb-5806750659e5/extractor/0.log"
Jul 30 21:28:25.611833 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:25.611806    2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9kzmr/perf-node-gather-daemonset-j8mx8"
Jul 30 21:28:26.327358 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:26.327339    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dldl8_63b4f0a2-c8a6-40a6-b00f-cd981d354b26/server/0.log"
Jul 30 21:28:26.396608 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:26.396577    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8nzgh_fd289d15-b395-4a6b-94e2-36500578a89e/manager/0.log"
Jul 30 21:28:26.469753 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:26.469731    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fd799f58b-skt94_71fe5fe6-416c-4479-8fa0-be872a6157e5/manager/0.log"
Jul 30 21:28:26.491677 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:26.491654    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fd799f58b-skt94_71fe5fe6-416c-4479-8fa0-be872a6157e5/copy-manifests/0.log"
Jul 30 21:28:28.033892 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:28.033864    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ccnsn_b944a48c-5904-4e31-b715-765956c18303/discovery/0.log"
Jul 30 21:28:28.045039 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:28.045017    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66684cbb98-8mqzf_a1996729-7f1a-45fe-8544-fc47129da3c2/kube-auth-proxy/0.log"
Jul 30 21:28:28.125769 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:28.125694    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-56rxl_74572307-3be4-4d10-a667-a7ccfbf839ba/payload-pre-processing/0.log"
Jul 30 21:28:28.168534 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:28.168508    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68cfb99b6d-9cgw6_b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3/router/0.log"
Jul 30 21:28:29.752791 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:29.752695    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ccnsn_b944a48c-5904-4e31-b715-765956c18303/discovery/0.log"
Jul 30 21:28:29.769465 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:29.769446    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66684cbb98-8mqzf_a1996729-7f1a-45fe-8544-fc47129da3c2/kube-auth-proxy/0.log"
Jul 30 21:28:29.841024 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:29.841006    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-56rxl_74572307-3be4-4d10-a667-a7ccfbf839ba/payload-pre-processing/0.log"
Jul 30 21:28:29.882593 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:29.882572    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68cfb99b6d-9cgw6_b170d49a-a3be-41d4-b0b4-97c3d3f4a8d3/router/0.log"
Jul 30 21:28:34.424941 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:34.424914    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2s59d_48cbf098-a8a1-4689-9a3c-ea8784346402/kube-multus-additional-cni-plugins/0.log"
Jul 30 21:28:34.447794 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:34.447754    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2s59d_48cbf098-a8a1-4689-9a3c-ea8784346402/egress-router-binary-copy/0.log"
Jul 30 21:28:34.470377 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:34.470360    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2s59d_48cbf098-a8a1-4689-9a3c-ea8784346402/cni-plugins/0.log"
Jul 30 21:28:34.491653 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:34.491638    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2s59d_48cbf098-a8a1-4689-9a3c-ea8784346402/bond-cni-plugin/0.log"
Jul 30 21:28:34.513254 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:34.513236    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2s59d_48cbf098-a8a1-4689-9a3c-ea8784346402/routeoverride-cni/0.log"
Jul 30 21:28:34.534720 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:34.534681    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2s59d_48cbf098-a8a1-4689-9a3c-ea8784346402/whereabouts-cni-bincopy/0.log"
Jul 30 21:28:34.555510 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:34.555491    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2s59d_48cbf098-a8a1-4689-9a3c-ea8784346402/whereabouts-cni/0.log"
Jul 30 21:28:34.962678 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:34.962623    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jvrq2_6db055c7-8e05-44de-8bc5-716d2b6957f2/kube-multus/0.log"
Jul 30 21:28:35.028231 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:35.028207    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l6rqg_d5679c74-1e69-4be8-90d8-3bf6aa168ee4/network-metrics-daemon/0.log"
Jul 30 21:28:35.049890 ip-10-0-134-148 kubenswrapper[2547]: I0730 21:28:35.049867    2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l6rqg_d5679c74-1e69-4be8-90d8-3bf6aa168ee4/kube-rbac-proxy/0.log"