Jul 21 12:09:07.319665 ip-10-0-138-237 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 21 12:09:07.319683 ip-10-0-138-237 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 21 12:09:07.319693 ip-10-0-138-237 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 21 12:09:07.320050 ip-10-0-138-237 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 21 12:09:17.476300 ip-10-0-138-237 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 21 12:09:17.476317 ip-10-0-138-237 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 61a2d86c401643a78130c12119f04209 --
Jul 21 12:11:14.615181 ip-10-0-138-237 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 12:11:14.945305 ip-10-0-138-237 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:11:14.945305 ip-10-0-138-237 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 12:11:14.945305 ip-10-0-138-237 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:11:14.945305 ip-10-0-138-237 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 12:11:14.945305 ip-10-0-138-237 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:11:14.946886 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.946772    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 12:11:14.952102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952085    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:11:14.952102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952101    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952105    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952108    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952113    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952116    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952119    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952123    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952126    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952129    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952132    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952135    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952138    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952140    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952143    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952146    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952149    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952151    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952154    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952158    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:11:14.952166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952162    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952172    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952176    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952179    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952182    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952190    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952193    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952196    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952198    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952201    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952203    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952206    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952208    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952211    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952213    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952216    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952220    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952223    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952225    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:11:14.952706 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952228    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952230    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952233    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952235    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952238    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952241    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952244    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952246    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952248    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952251    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952253    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952257    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952259    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952262    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952265    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952268    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952270    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952273    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952275    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952278    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:11:14.953193 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952281    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952283    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952285    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952288    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952291    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952293    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952296    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952298    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952301    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952303    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952305    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952308    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952310    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952313    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952315    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952318    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952320    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952322    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952325    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:11:14.953698 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952327    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952330    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952332    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952334    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952337    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952340    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952342    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952345    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952348    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952351    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952353    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952356    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952358    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952360    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952363    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952365    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952367    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952370    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952372    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952375    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:11:14.954156 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.952377    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953128    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953135    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953138    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953141    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953143    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953146    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953149    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953151    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953155    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953157    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953160    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953162    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953165    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953167    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953169    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953173    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953176    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953178    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:11:14.954665 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953181    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953186    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953190    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953193    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953197    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953200    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953204    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953207    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953210    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953214    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953216    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953219    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953221    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953224    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953226    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953229    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953231    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953234    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953236    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953239    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:11:14.955144 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953241    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953244    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953246    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953249    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953251    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953253    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953256    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953258    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953260    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953264    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953267    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953269    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953272    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953275    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953277    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953280    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953282    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953285    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953287    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:11:14.955669 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953290    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953293    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953295    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953297    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953301    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953305    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953308    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953310    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953313    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953316    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953319    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953321    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953324    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953326    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953329    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953331    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953334    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953336    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953338    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953341    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:11:14.956166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953343    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953345    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953348    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953351    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953353    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953357    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953359    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953362    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953364    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953367    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953370    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953372    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953375    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953378    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953380    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953384    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953386    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953390    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953393    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953397    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:11:14.956643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953400    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953402    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953765    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953770    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953772    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953775    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953778    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953780    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953783    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953786    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953805    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953810    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953814    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953817    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953820    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953822    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953825    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953827    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953830    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953832    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:11:14.957159 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953835    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953838    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953842    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953845    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953847    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953850    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953852    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953855    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953857    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953860    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953863    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953866    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953870    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953873    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953876    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953879    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953881    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953883    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953886    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:11:14.957635 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953889    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953892    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953894    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953897    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953899    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953902    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953904    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953907    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953911    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953914    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953917    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953920    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953923    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953926    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953928    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953931    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953933    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953936    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953939    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953942    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:11:14.958102 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953944    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953947    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953949    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953951    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953954    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953956    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953959    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953961    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953964    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953966    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953969    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953972    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953974    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953978    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953980    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953983    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953986    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953989    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953991    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953994    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:11:14.958581 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953996    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.953999    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954001    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954004    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954006    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954009    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954011    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954014    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954017    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954020    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954022    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954025    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954027    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954030    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954032    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954034    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954037    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954040    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954042    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954047    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:11:14.959077 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954049    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954052    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954124    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954131    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954137    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954144    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954148    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954151    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954156    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954160    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954163    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954167    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954170    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954173    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954176    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954179    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954182    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954185    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954188    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954191    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954194    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954197    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954200    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954203    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 12:11:14.959563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954207    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954210    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954213    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954216    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954219    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954223    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954225    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954229    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954232    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954235    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954239    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954242    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954245    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954249    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954252    2562 flags.go:64] FLAG: --enable-server="true"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954254    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954259    2562 flags.go:64] FLAG: --event-burst="100"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954261    2562 flags.go:64] FLAG: --event-qps="50"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954265    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954268    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954271    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954274    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954277    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954280    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954282    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 21 12:11:14.960186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954285    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954288    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954291    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954294    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954297    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954299    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954303    2562 flags.go:64] FLAG: --feature-gates=""
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954309    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954312    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954315    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954318    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954321    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954324    2562 flags.go:64] FLAG: --help="false"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954326    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-138-237.ec2.internal"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954331    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954333    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954336    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954339    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954343    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954345    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954349    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954352    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954354    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 12:11:14.960839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954357    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954360    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954363    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954366    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954368    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954371    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954374    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954377    2562 flags.go:64] FLAG: --lock-file=""
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954379    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954382    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954385    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954390    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954393    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954396    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954399    2562 flags.go:64] FLAG: --logging-format="text"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954401    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954405    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954408    2562 flags.go:64] FLAG: --manifest-url=""
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954411    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954416    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954418    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954422    2562 flags.go:64] FLAG: --max-pods="110"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954425    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954429    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954432    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 12:11:14.961469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954434    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954438    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954440    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954443    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954451    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954454    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954457    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954460    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954463    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954468    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954471    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954474    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954477    2562 flags.go:64] FLAG: --port="10250"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954480    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954483    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bc86078a5b4587df"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954486    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954488    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954491    2562 flags.go:64] FLAG: --register-node="true"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954494    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954497    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954501    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954503    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954506    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954509    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954513    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 12:11:14.962321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954516    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954518    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954521    2562 flags.go:64] FLAG: --runonce="false"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954523    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954526    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954529    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954534    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954537    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954539    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954542    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954545    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954550    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954553    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954555    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954558    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954561    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954564    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954566    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954571    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954574    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954577    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954580    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954583    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954585    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954588    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 12:11:14.963164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954591    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954594    2562 flags.go:64] FLAG: --v="2"
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954597    2562 flags.go:64] FLAG: --version="false"
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954601    2562 flags.go:64] FLAG: --vmodule=""
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954605    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.954608    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954724    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954728    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954731    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954734    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954737    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954740    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954742    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954746    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954750    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954754    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954757    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954759    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954763    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954765    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:11:14.964222 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954769    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954772    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954775    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954778    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954781    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954784    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954786    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954803    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954806    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954808    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954811    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954813    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954816    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954819    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954821    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954824    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954826    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954829    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954831    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954834    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:11:14.964915 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954836    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954839    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954842    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954844    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954847    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954850    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954853    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954856    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954858    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954861    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954864    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954866    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954869    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954872    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954874    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954877    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954879    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954882    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954884    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954887    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:11:14.965627 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954889    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954891    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954894    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954896    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954899    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954901    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954904    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954906    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954908    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954911    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954914    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954916    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954919    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954922    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954924    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954927    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954930    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954933    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954936    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954938    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:11:14.966504 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954941    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954943    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954946    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954948    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954950    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954953    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954955    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954958    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954960    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954963    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954965    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954968    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954970    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954972    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954975    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954977    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954980    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954982    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954985    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954987    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:11:14.967130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954990    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954992    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954996    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.954999    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.955001    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.955010    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.964575    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.964595    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.965229    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.965247    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965337    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965346    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965351    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965356    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:11:14.967896 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965360    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965365    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965370    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965375    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965379    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965383    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965388    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965392    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965397    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965402    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965406    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965412    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965416    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965420    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965427    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965434    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965438    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965443    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965447    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965452    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:11:14.968251 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965456    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965460    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965464    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965469    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965473    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965478    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965482    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965487    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965491    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965495    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965500    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965504    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965509    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965512    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965517    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965521    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965525    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965530    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965534    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965539    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:11:14.968912 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965545    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965551    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965565    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965570    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965574    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965578    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965583    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965587    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965591    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965595    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965599    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965603    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965607    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965611    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965615    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965621    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965626    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965630    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965634    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965639    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:11:14.969686 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965645    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965650    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965656    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965661    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965665    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965669    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965673    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965678    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965682    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965686    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965690    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965695    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965698    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965703    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965707    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965719    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965724    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965729    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965733    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965737    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:11:14.970316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965741    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965746    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965750    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965754    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965759    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965766    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965771    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965776    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965781    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965786    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965810    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965815    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965819    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965824    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:11:14.970862 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.965828    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.965837    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966026    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966035    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966040    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966045    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966049    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966053    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966057    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966061    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966066    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966070    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966074    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966079    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966085    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:11:14.971216 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966089    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966093    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966098    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966102    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966106    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966110    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966114    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966118    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966122    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966126    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966131    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966135    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966142    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966149    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966154    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966158    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966162    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966167    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966171    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:11:14.971646 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966175    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966179    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966183    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966187    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966192    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966196    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966200    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966204    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966208    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966212    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966216    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966221    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966225    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966230    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966234    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966238    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966242    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966246    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966250    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966254    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:11:14.972139 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966257    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966261    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966266    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966270    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966274    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966279    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966283    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966287    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966291    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966295    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966299    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966304    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966308    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966314    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966320    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966324    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966329    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966333    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966337    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966341    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:11:14.972701 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966345    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966349    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966354    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966358    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966362    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966366    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966371    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966376    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966380    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966385    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966389    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966393    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966397    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966401    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966405    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966409    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966413    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966420    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966425    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:11:14.973348 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966430    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966434    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966438    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966443    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966447    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966451    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966455    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:14.966458    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.966466    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.966657    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.970145    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.971410    2562 server.go:1019] "Starting client certificate rotation"
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.971517    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 12:11:14.973959 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.971548    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 12:11:14.974814 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.974776    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 12:11:14.976030 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.976007    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 12:11:14.985489 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.985474    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 21 12:11:14.988150 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.988138    2562 log.go:25] "Validated CRI v1 image API"
Jul 21 12:11:14.989623 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.989603    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 12:11:14.991816 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.991781    2562 fs.go:135] Filesystem UUIDs: map[26d9a145-383b-4c66-a4bd-2391edf31c6c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e6cf4082-92a7-4371-b3d4-2969989a43bd:/dev/nvme0n1p3]
Jul 21 12:11:14.991858 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.991819    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 12:11:14.996759 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.996737    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 12:11:14.997834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.997707    2562 manager.go:216] Machine: {Timestamp:2026-07-21 12:11:14.995986364 +0000 UTC m=+0.279399657 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099369 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec204cb5d29c55bb714caae1fc6d77a0 SystemUUID:ec204cb5-d29c-55bb-714c-aae1fc6d77a0 BootID:61a2d86c-4016-43a7-8130-c12119f04209 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cb:7c:14:7d:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cb:7c:14:7d:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:24:8c:13:15:e9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 12:11:14.997834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.997832    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 12:11:14.997943 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:14.997916    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 12:11:15.000627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.000603    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 12:11:15.000771 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.000630    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-237.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 12:11:15.001782 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.001770    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 12:11:15.001827 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.001785    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 12:11:15.001827 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.001810    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 12:11:15.002417 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.002406    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 12:11:15.003887 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.003876    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 12:11:15.007628 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.007616    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 12:11:15.007765 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.007755    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 12:11:15.007815 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.007802    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 12:11:15.007852 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.007818    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 12:11:15.007852 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.007842    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 12:11:15.007908 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.007855    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 21 12:11:15.007908 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.007867    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 12:11:15.008313 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.008302    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 12:11:15.008352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.008325    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 12:11:15.010172 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.010156    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 12:11:15.011943 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.011926    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 12:11:15.012033 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.011965    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 12:11:15.013285 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013270    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 12:11:15.013355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013291    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 12:11:15.013355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013301    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 12:11:15.013355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013309    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 12:11:15.013355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013321    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 12:11:15.013355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013329    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 12:11:15.013355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013340    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 12:11:15.013355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013349    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 12:11:15.013355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013358    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 12:11:15.013597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013367    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 12:11:15.013597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013386    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 12:11:15.013597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.013399    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 12:11:15.014038 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.014026    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 12:11:15.014095 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.014041    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 12:11:15.016447 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.016428    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 12:11:15.016447 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.016424    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 12:11:15.016560 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.016496    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 12:11:15.017354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.017342    2562 server.go:1268] "Started kubelet"
Jul 21 12:11:15.017432 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.017406    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 12:11:15.017514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.017476    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 12:11:15.017560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.017551    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 12:11:15.018450 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.018421    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6ztlw"
Jul 21 12:11:15.019290 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.019274    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 12:11:15.019565 ip-10-0-138-237 systemd[1]: Started Kubernetes Kubelet.
Jul 21 12:11:15.020377 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.020290    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 12:11:15.023172 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.023154    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 12:11:15.024695 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.023906    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-237.ec2.internal.18c44cdf77d33cc3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-237.ec2.internal,UID:ip-10-0-138-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-237.ec2.internal,},FirstTimestamp:2026-07-21 12:11:15.017309379 +0000 UTC m=+0.300722662,LastTimestamp:2026-07-21 12:11:15.017309379 +0000 UTC m=+0.300722662,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-237.ec2.internal,}"
Jul 21 12:11:15.025528 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.025513    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 12:11:15.025582 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.025531    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 12:11:15.025642 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.025585    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.026113 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026092    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 12:11:15.026203 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026149    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 12:11:15.026203 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026198    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 12:11:15.026352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026327    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 12:11:15.026352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026337    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 12:11:15.026686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026666    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6ztlw"
Jul 21 12:11:15.026889 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026860    2562 factory.go:153] Registering CRI-O factory
Jul 21 12:11:15.026889 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026876    2562 factory.go:223] Registration of the crio container factory successfully
Jul 21 12:11:15.026976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026923    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 12:11:15.026976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026951    2562 factory.go:55] Registering systemd factory
Jul 21 12:11:15.026976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026957    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 21 12:11:15.026976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026973    2562 factory.go:103] Registering Raw factory
Jul 21 12:11:15.027151 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.026983    2562 manager.go:1196] Started watching for new ooms in manager
Jul 21 12:11:15.027534 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.027522    2562 manager.go:349] Starting recovery of all containers
Jul 21 12:11:15.028878 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.028845    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 12:11:15.029084 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.029057    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 12:11:15.040767 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.040608    2562 manager.go:354] Recovery completed
Jul 21 12:11:15.041964 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.041947    2562 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 21 12:11:15.044941 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.044849    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:11:15.046714 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.046695    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:11:15.046779 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.046729    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:11:15.046779 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.046740    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:11:15.047276 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.047258    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 12:11:15.047276 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.047272    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 12:11:15.047276 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.047275    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:11:15.047470 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.047290    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 12:11:15.048014 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.048001    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:11:15.048066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.048030    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:11:15.048066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.048040    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:11:15.049485 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.049471    2562 policy_none.go:49] "None policy: Start"
Jul 21 12:11:15.049553 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.049490    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 12:11:15.049553 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.049503    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 12:11:15.051522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.051508    2562 policy_none.go:47] "Start"
Jul 21 12:11:15.090359 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.090344    2562 manager.go:337] "Starting Device Plugin manager"
Jul 21 12:11:15.090448 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.090380    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 12:11:15.090448 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.090392    2562 server.go:87] "Starting device plugin registration server"
Jul 21 12:11:15.090641 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.090628    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 12:11:15.090692 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.090642    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 12:11:15.090756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.090740    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 12:11:15.091046 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.090849    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 12:11:15.091046 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.090861    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 12:11:15.091378 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.091356    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 12:11:15.091490 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.091416    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.184299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.184259    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 12:11:15.186450 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.185627    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 12:11:15.186450 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.185662    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 12:11:15.186450 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.185683    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 12:11:15.186450 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.185718    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 12:11:15.188560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.188541    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:11:15.190891 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.190878    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:11:15.191705 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.191690    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:11:15.191764 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.191719    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:11:15.191764 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.191730    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:11:15.191764 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.191764    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.200276 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.200230    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.200276 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.200253    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-237.ec2.internal\": node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.216478 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.216457    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.286326 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.286291    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal"]
Jul 21 12:11:15.286469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.286390    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:11:15.287331 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.287317    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:11:15.287423 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.287344    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:11:15.287423 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.287358    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:11:15.289703 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.289690    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:11:15.289850 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.289835    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.289911 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.289870    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:11:15.290416 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.290401    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:11:15.290416 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.290410    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:11:15.290533 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.290427    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:11:15.290533 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.290439    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:11:15.290533 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.290430    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:11:15.290533 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.290505    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:11:15.292677 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.292664    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.292738 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.292687    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:11:15.293345 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.293328    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:11:15.293441 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.293353    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:11:15.293441 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.293362    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:11:15.306337 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.306318    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-237.ec2.internal\" not found" node="ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.310356 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.310340    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-237.ec2.internal\" not found" node="ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.317379 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.317365    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.327895 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.327876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/677db6ca0d35088a971975beecb717c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"677db6ca0d35088a971975beecb717c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.327989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.327902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/677db6ca0d35088a971975beecb717c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"677db6ca0d35088a971975beecb717c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.327989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.327927    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/378731304c8222449ffad4ab0f72e94a-config\") pod \"kube-apiserver-proxy-ip-10-0-138-237.ec2.internal\" (UID: \"378731304c8222449ffad4ab0f72e94a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.417526 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.417496    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.428881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.428862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/677db6ca0d35088a971975beecb717c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"677db6ca0d35088a971975beecb717c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.428951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.428895    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/677db6ca0d35088a971975beecb717c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"677db6ca0d35088a971975beecb717c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.428951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.428914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/378731304c8222449ffad4ab0f72e94a-config\") pod \"kube-apiserver-proxy-ip-10-0-138-237.ec2.internal\" (UID: \"378731304c8222449ffad4ab0f72e94a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.429047 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.428957    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/677db6ca0d35088a971975beecb717c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"677db6ca0d35088a971975beecb717c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.429047 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.428971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/677db6ca0d35088a971975beecb717c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal\" (UID: \"677db6ca0d35088a971975beecb717c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.429047 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.428966    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/378731304c8222449ffad4ab0f72e94a-config\") pod \"kube-apiserver-proxy-ip-10-0-138-237.ec2.internal\" (UID: \"378731304c8222449ffad4ab0f72e94a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.518375 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.518292    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.609808 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.609762    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.613028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.613009    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:15.619338 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.619322    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.719808 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.719750    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.820331 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.820275    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.920803 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:15.920760    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:15.971274 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.971251    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 12:11:15.971746 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:15.971384    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 12:11:16.021042 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:16.021018    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:16.035833 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.035784    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 12:06:16 +0000 UTC" deadline="2028-04-11 11:53:47.002019994 +0000 UTC"
Jul 21 12:11:16.035941 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.035832    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15119h42m30.966193047s"
Jul 21 12:11:16.121256 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:16.121196    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:16.121572 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.121558    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:11:16.186013 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:16.185984    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod378731304c8222449ffad4ab0f72e94a.slice/crio-957b4a0873d1644e21d653ff6f2a479e68b98768c2a1d18f04f2bec67a7d36f9 WatchSource:0}: Error finding container 957b4a0873d1644e21d653ff6f2a479e68b98768c2a1d18f04f2bec67a7d36f9: Status 404 returned error can't find the container with id 957b4a0873d1644e21d653ff6f2a479e68b98768c2a1d18f04f2bec67a7d36f9
Jul 21 12:11:16.186258 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:16.186240    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod677db6ca0d35088a971975beecb717c8.slice/crio-6b07067ce88ade71fea1be5ae00d8ecdd769d3d46c8872ee9de7cbdb73f977d0 WatchSource:0}: Error finding container 6b07067ce88ade71fea1be5ae00d8ecdd769d3d46c8872ee9de7cbdb73f977d0: Status 404 returned error can't find the container with id 6b07067ce88ade71fea1be5ae00d8ecdd769d3d46c8872ee9de7cbdb73f977d0
Jul 21 12:11:16.191041 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.191026    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:11:16.221727 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:16.221703    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:16.322258 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:16.322230    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-237.ec2.internal\" not found"
Jul 21 12:11:16.379344 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.379280    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:11:16.411640 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.411617    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:11:16.426724 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.426703    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:16.438156 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.438136    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 12:11:16.438955 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.438944    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal"
Jul 21 12:11:16.447781 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:16.447768    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 12:11:18.164655 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.164623    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:11:18.233889 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.233859    2562 apiserver.go:52] "Watching apiserver"
Jul 21 12:11:18.240806 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.240767    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 12:11:18.242759 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.242730    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal","openshift-multus/network-metrics-daemon-4tb74","openshift-ovn-kubernetes/ovnkube-node-hbwbw","kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal","openshift-cluster-node-tuning-operator/tuned-kbcz9","openshift-dns/node-resolver-75gck","openshift-image-registry/node-ca-vfgrc","openshift-multus/multus-7h88j","openshift-multus/multus-additional-cni-plugins-5jnqf","openshift-network-diagnostics/network-check-target-lpnp2","openshift-network-operator/iptables-alerter-d4dlj","kube-system/global-pull-secret-syncer-q7tnn","kube-system/konnectivity-agent-c2ltb"]
Jul 21 12:11:18.245498 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.245472    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.247852 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.247827    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:18.247953 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.247900    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:18.248007 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.247968    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 12:11:18.248479 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.248459    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 12:11:18.248689 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.248672    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kl9lt\""
Jul 21 12:11:18.250952 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.250933    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 12:11:18.251808 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.251163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:18.251808 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.251245    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:18.254918 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.254899    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.257123 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.257096    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 12:11:18.257666 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.257646    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 12:11:18.257970 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.257952    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 12:11:18.258191 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.258174    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 12:11:18.258369 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.258351    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 12:11:18.258564 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.258549    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 12:11:18.258843 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.258824    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6b6b4\""
Jul 21 12:11:18.259262 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.259244    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:18.260273 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.260253    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.261398 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.261372    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 12:11:18.261710 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.261691    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t8dm4\""
Jul 21 12:11:18.263178 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.261931    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 12:11:18.263178 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.262200    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 12:11:18.263178 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.262471    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kcwn9\""
Jul 21 12:11:18.263178 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.262599    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:11:18.263178 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.262757    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 12:11:18.263178 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.262846    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.263962 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.263933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264062 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.263992    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/24bc8242-dbb7-49de-a524-4c1ec938353b-ovn-node-metrics-cert\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264062 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-systemd-units\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264062 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264056    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-kubelet\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264210 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264102    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163cac4c-f48f-41af-9e32-faf67720471d-tmp-dir\") pod \"node-resolver-75gck\" (UID: \"163cac4c-f48f-41af-9e32-faf67720471d\") " pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.264210 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264125    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-slash\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264210 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-run-netns\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264210 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/24bc8242-dbb7-49de-a524-4c1ec938353b-env-overrides\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264228    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/163cac4c-f48f-41af-9e32-faf67720471d-hosts-file\") pod \"node-resolver-75gck\" (UID: \"163cac4c-f48f-41af-9e32-faf67720471d\") " pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.264413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnvtd\" (UniqueName: \"kubernetes.io/projected/163cac4c-f48f-41af-9e32-faf67720471d-kube-api-access-dnvtd\") pod \"node-resolver-75gck\" (UID: \"163cac4c-f48f-41af-9e32-faf67720471d\") " pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.264413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/24bc8242-dbb7-49de-a524-4c1ec938353b-ovnkube-script-lib\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-run-ovn\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264339    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-node-log\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-log-socket\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/24bc8242-dbb7-49de-a524-4c1ec938353b-ovnkube-config\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264418    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx2bc\" (UniqueName: \"kubernetes.io/projected/24bc8242-dbb7-49de-a524-4c1ec938353b-kube-api-access-xx2bc\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264435    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264453    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.264497    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264516    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncw7l\" (UniqueName: \"kubernetes.io/projected/fa35c26d-86c4-456a-862f-fdc14069ec25-kube-api-access-ncw7l\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264564    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-run-openvswitch\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264623    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-cni-netd\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.264667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264660    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-run-systemd\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.265158 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-etc-openvswitch\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.265158 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-var-lib-openvswitch\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.265158 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.264733    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-cni-bin\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.265158 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.265064    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 12:11:18.265158 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.265102    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 12:11:18.265414 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.265404    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 12:11:18.265688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.265671    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 12:11:18.265809 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.265792    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-j8dgf\""
Jul 21 12:11:18.266803 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.266712    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.268733 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.268714    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 12:11:18.269525 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.269506    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 12:11:18.270641 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.270576    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4572f\""
Jul 21 12:11:18.271015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.270996    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.271767 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.271747    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 12:11:18.274369 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.274200    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 12:11:18.274369 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.274262    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 12:11:18.274369 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.274206    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z2tz5\""
Jul 21 12:11:18.274877 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.274854    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.277139 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.277121    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87qfx\""
Jul 21 12:11:18.277382 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.277357    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 12:11:18.277382 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.277377    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 12:11:18.277509 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.277381    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.277588 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.277565    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 12:11:18.279822 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.279804    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 12:11:18.280017 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.279998    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:11:18.280123 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.280047    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 12:11:18.280194 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.280163    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cwkrs\""
Jul 21 12:11:18.291008 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.290991    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f797m"
Jul 21 12:11:18.301125 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.301105    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f797m"
Jul 21 12:11:18.352316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.352275    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 12:11:18.364897 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.364870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-cnibin\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.364897 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.364899    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-os-release\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.365085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.364916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-socket-dir-parent\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.365085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.364937    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-var-lib-cni-bin\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.365085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.364966    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh67x\" (UniqueName: \"kubernetes.io/projected/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-kube-api-access-vh67x\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.365085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.364998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-systemd-units\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365023    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-kubelet\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365067    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-systemd-units\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-sysctl-d\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.365324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365146    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-sysctl-conf\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.365324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365163    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-kubelet\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wplkk\" (UniqueName: \"kubernetes.io/projected/950932a8-0a9d-4ba6-ba69-e9310263a09a-kube-api-access-wplkk\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.365324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365209    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163cac4c-f48f-41af-9e32-faf67720471d-tmp-dir\") pod \"node-resolver-75gck\" (UID: \"163cac4c-f48f-41af-9e32-faf67720471d\") " pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.365324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/24bc8242-dbb7-49de-a524-4c1ec938353b-env-overrides\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365262    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-var-lib-kubelet\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.365324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-run-multus-certs\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.365324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365313    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-cnibin\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/163cac4c-f48f-41af-9e32-faf67720471d-hosts-file\") pod \"node-resolver-75gck\" (UID: \"163cac4c-f48f-41af-9e32-faf67720471d\") " pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvtd\" (UniqueName: \"kubernetes.io/projected/163cac4c-f48f-41af-9e32-faf67720471d-kube-api-access-dnvtd\") pod \"node-resolver-75gck\" (UID: \"163cac4c-f48f-41af-9e32-faf67720471d\") " pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365392    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/24bc8242-dbb7-49de-a524-4c1ec938353b-ovnkube-script-lib\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365418    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvkxh\" (UniqueName: \"kubernetes.io/projected/b848d323-22e3-44fa-a82b-2994f66c296a-kube-api-access-kvkxh\") pod \"node-ca-vfgrc\" (UID: \"b848d323-22e3-44fa-a82b-2994f66c296a\") " pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365441    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-socket-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-var-lib-cni-multus\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365517    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163cac4c-f48f-41af-9e32-faf67720471d-tmp-dir\") pod \"node-resolver-75gck\" (UID: \"163cac4c-f48f-41af-9e32-faf67720471d\") " pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-run-ovn\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365580    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-log-socket\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365589    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-run-ovn\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365619    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/24bc8242-dbb7-49de-a524-4c1ec938353b-ovnkube-config\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh682\" (UniqueName: \"kubernetes.io/projected/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-kube-api-access-nh682\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-sysconfig\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365665    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-systemd\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365688    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365713    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/24bc8242-dbb7-49de-a524-4c1ec938353b-env-overrides\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.365810 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365723    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365755    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncw7l\" (UniqueName: \"kubernetes.io/projected/fa35c26d-86c4-456a-862f-fdc14069ec25-kube-api-access-ncw7l\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365805    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-run-openvswitch\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365835    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-sys-fs\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365866    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-etc-kubernetes\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b6c7079-8edc-469b-af1c-61f5ccfa0511-dbus\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365964    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ed56e16-95f6-4ab2-be2d-2e1798e82604-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.365992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-run-systemd\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366017    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-cni-dir\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366034    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-log-socket\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366050    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-daemon-config\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366076    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ed56e16-95f6-4ab2-be2d-2e1798e82604-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh74k\" (UniqueName: \"kubernetes.io/projected/0ed56e16-95f6-4ab2-be2d-2e1798e82604-kube-api-access-jh74k\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-var-lib-openvswitch\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/24bc8242-dbb7-49de-a524-4c1ec938353b-ovnkube-script-lib\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/24bc8242-dbb7-49de-a524-4c1ec938353b-ovnkube-config\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.366219    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:18.366637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366228    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-var-lib-openvswitch\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/950932a8-0a9d-4ba6-ba69-e9310263a09a-tmp\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-run-systemd\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.366303    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs podName:fa35c26d-86c4-456a-862f-fdc14069ec25 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:18.866266759 +0000 UTC m=+2.924697760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs") pod "network-metrics-daemon-4tb74" (UID: "fa35c26d-86c4-456a-862f-fdc14069ec25") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366315    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-run-openvswitch\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-system-cni-dir\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366352    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b6c7079-8edc-469b-af1c-61f5ccfa0511-kubelet-config\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366418    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/163cac4c-f48f-41af-9e32-faf67720471d-hosts-file\") pod \"node-resolver-75gck\" (UID: \"163cac4c-f48f-41af-9e32-faf67720471d\") " pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366457    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn6l2\" (UniqueName: \"kubernetes.io/projected/f63d5f94-73d9-4236-9ed5-3f6fa4da022a-kube-api-access-wn6l2\") pod \"iptables-alerter-d4dlj\" (UID: \"f63d5f94-73d9-4236-9ed5-3f6fa4da022a\") " pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-kubernetes\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366530    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-run\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366552    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-tuned\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366620    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-slash\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.367376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366637    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-run-netns\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b848d323-22e3-44fa-a82b-2994f66c296a-serviceca\") pod \"node-ca-vfgrc\" (UID: \"b848d323-22e3-44fa-a82b-2994f66c296a\") " pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ed56e16-95f6-4ab2-be2d-2e1798e82604-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366726    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f63d5f94-73d9-4236-9ed5-3f6fa4da022a-iptables-alerter-script\") pod \"iptables-alerter-d4dlj\" (UID: \"f63d5f94-73d9-4236-9ed5-3f6fa4da022a\") " pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366742    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-host\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-system-cni-dir\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366796    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-os-release\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366846    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-node-log\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-slash\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-run-netns\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366887    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx2bc\" (UniqueName: \"kubernetes.io/projected/24bc8242-dbb7-49de-a524-4c1ec938353b-kube-api-access-xx2bc\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366896    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-node-log\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-run-k8s-cni-cncf-io\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.366970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-run-netns\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367036    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-hostroot\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-sys\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-lib-modules\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367101    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-cni-netd\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-etc-openvswitch\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b848d323-22e3-44fa-a82b-2994f66c296a-host\") pod \"node-ca-vfgrc\" (UID: \"b848d323-22e3-44fa-a82b-2994f66c296a\") " pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367192    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f63d5f94-73d9-4236-9ed5-3f6fa4da022a-host-slash\") pod \"iptables-alerter-d4dlj\" (UID: \"f63d5f94-73d9-4236-9ed5-3f6fa4da022a\") " pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367205    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-modprobe-d\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-etc-openvswitch\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-cni-binary-copy\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367246    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367251    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367267    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-cni-netd\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367290    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-cni-bin\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367319    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/24bc8242-dbb7-49de-a524-4c1ec938353b-ovn-node-metrics-cert\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367366    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24bc8242-dbb7-49de-a524-4c1ec938353b-host-cni-bin\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-device-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-etc-selinux\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70679208-3538-46cc-8b71-0e27edb8344c-agent-certs\") pod \"konnectivity-agent-c2ltb\" (UID: \"70679208-3538-46cc-8b71-0e27edb8344c\") " pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:18.368769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367504    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-var-lib-kubelet\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.369387 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-conf-dir\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.369387 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.369387 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-registration-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.369387 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367602    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70679208-3538-46cc-8b71-0e27edb8344c-konnectivity-ca\") pod \"konnectivity-agent-c2ltb\" (UID: \"70679208-3538-46cc-8b71-0e27edb8344c\") " pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:18.369387 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.367688    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 12:11:18.371243 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.371223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/24bc8242-dbb7-49de-a524-4c1ec938353b-ovn-node-metrics-cert\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.371530 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.371509    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:11:18.371530 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.371532    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:11:18.371677 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.371543    2562 projected.go:196] Error preparing data for projected volume kube-api-access-78248 for pod openshift-network-diagnostics/network-check-target-lpnp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:18.371677 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.371599    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248 podName:b788c24c-d635-4fcb-9f86-4ae4b2a1f56d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:18.871582291 +0000 UTC m=+2.930013290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-78248" (UniqueName: "kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248") pod "network-check-target-lpnp2" (UID: "b788c24c-d635-4fcb-9f86-4ae4b2a1f56d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:18.373467 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.373449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnvtd\" (UniqueName: \"kubernetes.io/projected/163cac4c-f48f-41af-9e32-faf67720471d-kube-api-access-dnvtd\") pod \"node-resolver-75gck\" (UID: \"163cac4c-f48f-41af-9e32-faf67720471d\") " pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.382740 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.382708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncw7l\" (UniqueName: \"kubernetes.io/projected/fa35c26d-86c4-456a-862f-fdc14069ec25-kube-api-access-ncw7l\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:18.383043 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.383024    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx2bc\" (UniqueName: \"kubernetes.io/projected/24bc8242-dbb7-49de-a524-4c1ec938353b-kube-api-access-xx2bc\") pod \"ovnkube-node-hbwbw\" (UID: \"24bc8242-dbb7-49de-a524-4c1ec938353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.416361 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.416249    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" event={"ID":"378731304c8222449ffad4ab0f72e94a","Type":"ContainerStarted","Data":"957b4a0873d1644e21d653ff6f2a479e68b98768c2a1d18f04f2bec67a7d36f9"}
Jul 21 12:11:18.417401 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.417375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" event={"ID":"677db6ca0d35088a971975beecb717c8","Type":"ContainerStarted","Data":"6b07067ce88ade71fea1be5ae00d8ecdd769d3d46c8872ee9de7cbdb73f977d0"}
Jul 21 12:11:18.468118 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-etc-selinux\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.468118 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70679208-3538-46cc-8b71-0e27edb8344c-agent-certs\") pod \"konnectivity-agent-c2ltb\" (UID: \"70679208-3538-46cc-8b71-0e27edb8344c\") " pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:18.468354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468138    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-var-lib-kubelet\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.468354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-conf-dir\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468185    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.468354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-registration-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.468354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70679208-3538-46cc-8b71-0e27edb8344c-konnectivity-ca\") pod \"konnectivity-agent-c2ltb\" (UID: \"70679208-3538-46cc-8b71-0e27edb8344c\") " pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:18.468354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-etc-selinux\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.468354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-var-lib-kubelet\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.468354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468343    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-cnibin\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468271    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-cnibin\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-os-release\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-socket-dir-parent\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468452    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-conf-dir\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-registration-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468500    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-socket-dir-parent\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468527    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-var-lib-cni-bin\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468555    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh67x\" (UniqueName: \"kubernetes.io/projected/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-kube-api-access-vh67x\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-os-release\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468584    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-sysctl-d\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-sysctl-conf\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468657    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wplkk\" (UniqueName: \"kubernetes.io/projected/950932a8-0a9d-4ba6-ba69-e9310263a09a-kube-api-access-wplkk\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-var-lib-kubelet\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-run-multus-certs\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-cnibin\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvkxh\" (UniqueName: \"kubernetes.io/projected/b848d323-22e3-44fa-a82b-2994f66c296a-kube-api-access-kvkxh\") pod \"node-ca-vfgrc\" (UID: \"b848d323-22e3-44fa-a82b-2994f66c296a\") " pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-socket-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.468855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468860    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-var-lib-cni-bin\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-run-multus-certs\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-socket-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-var-lib-cni-multus\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468954    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-cnibin\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh682\" (UniqueName: \"kubernetes.io/projected/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-kube-api-access-nh682\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.468990    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-sysconfig\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-systemd\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469062    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-sys-fs\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-etc-kubernetes\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b6c7079-8edc-469b-af1c-61f5ccfa0511-dbus\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ed56e16-95f6-4ab2-be2d-2e1798e82604-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-cni-dir\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-daemon-config\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-var-lib-kubelet\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469263    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-sysconfig\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-systemd\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.469686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-sys-fs\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469381    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-sysctl-d\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469386    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-etc-kubernetes\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-var-lib-cni-multus\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ed56e16-95f6-4ab2-be2d-2e1798e82604-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469550    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-cni-dir\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469559    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh74k\" (UniqueName: \"kubernetes.io/projected/0ed56e16-95f6-4ab2-be2d-2e1798e82604-kube-api-access-jh74k\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469593    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/950932a8-0a9d-4ba6-ba69-e9310263a09a-tmp\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469595    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5b6c7079-8edc-469b-af1c-61f5ccfa0511-dbus\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469673    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-sysctl-conf\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-system-cni-dir\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469749    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ed56e16-95f6-4ab2-be2d-2e1798e82604-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469754    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b6c7079-8edc-469b-af1c-61f5ccfa0511-kubelet-config\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469833    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5b6c7079-8edc-469b-af1c-61f5ccfa0511-kubelet-config\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469839    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-system-cni-dir\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469883    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn6l2\" (UniqueName: \"kubernetes.io/projected/f63d5f94-73d9-4236-9ed5-3f6fa4da022a-kube-api-access-wn6l2\") pod \"iptables-alerter-d4dlj\" (UID: \"f63d5f94-73d9-4236-9ed5-3f6fa4da022a\") " pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-kubernetes\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.470522 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.469921    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.469930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-run\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.469987    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret podName:5b6c7079-8edc-469b-af1c-61f5ccfa0511 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:18.969966288 +0000 UTC m=+3.028397272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret") pod "global-pull-secret-syncer-q7tnn" (UID: "5b6c7079-8edc-469b-af1c-61f5ccfa0511") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-tuned\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470035    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b848d323-22e3-44fa-a82b-2994f66c296a-serviceca\") pod \"node-ca-vfgrc\" (UID: \"b848d323-22e3-44fa-a82b-2994f66c296a\") " pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470059    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ed56e16-95f6-4ab2-be2d-2e1798e82604-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f63d5f94-73d9-4236-9ed5-3f6fa4da022a-iptables-alerter-script\") pod \"iptables-alerter-d4dlj\" (UID: \"f63d5f94-73d9-4236-9ed5-3f6fa4da022a\") " pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470107    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-host\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-multus-daemon-config\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470129    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-system-cni-dir\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-os-release\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-run-k8s-cni-cncf-io\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470206    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-run-netns\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-hostroot\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-sys\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470294    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-lib-modules\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470322    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b848d323-22e3-44fa-a82b-2994f66c296a-host\") pod \"node-ca-vfgrc\" (UID: \"b848d323-22e3-44fa-a82b-2994f66c296a\") " pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.471324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f63d5f94-73d9-4236-9ed5-3f6fa4da022a-host-slash\") pod \"iptables-alerter-d4dlj\" (UID: \"f63d5f94-73d9-4236-9ed5-3f6fa4da022a\") " pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-modprobe-d\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470390    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-cni-binary-copy\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70679208-3538-46cc-8b71-0e27edb8344c-konnectivity-ca\") pod \"konnectivity-agent-c2ltb\" (UID: \"70679208-3538-46cc-8b71-0e27edb8344c\") " pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-device-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470456    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ed56e16-95f6-4ab2-be2d-2e1798e82604-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470500    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-kubernetes\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-hostroot\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470580    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b848d323-22e3-44fa-a82b-2994f66c296a-serviceca\") pod \"node-ca-vfgrc\" (UID: \"b848d323-22e3-44fa-a82b-2994f66c296a\") " pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470591    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b848d323-22e3-44fa-a82b-2994f66c296a-host\") pod \"node-ca-vfgrc\" (UID: \"b848d323-22e3-44fa-a82b-2994f66c296a\") " pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470594    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-run\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-lib-modules\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-modprobe-d\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470649    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-host\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470658    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-system-cni-dir\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-device-dir\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/950932a8-0a9d-4ba6-ba69-e9310263a09a-sys\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.472026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470832    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f63d5f94-73d9-4236-9ed5-3f6fa4da022a-host-slash\") pod \"iptables-alerter-d4dlj\" (UID: \"f63d5f94-73d9-4236-9ed5-3f6fa4da022a\") " pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.472704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.472704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470887    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-run-k8s-cni-cncf-io\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.472704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470891    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-host-run-netns\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.472704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470902    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f63d5f94-73d9-4236-9ed5-3f6fa4da022a-iptables-alerter-script\") pod \"iptables-alerter-d4dlj\" (UID: \"f63d5f94-73d9-4236-9ed5-3f6fa4da022a\") " pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.472704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.470968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ed56e16-95f6-4ab2-be2d-2e1798e82604-os-release\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.472704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.471112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ed56e16-95f6-4ab2-be2d-2e1798e82604-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.472704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.471239    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-cni-binary-copy\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.472704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.472701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/950932a8-0a9d-4ba6-ba69-e9310263a09a-etc-tuned\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.473057 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.472745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70679208-3538-46cc-8b71-0e27edb8344c-agent-certs\") pod \"konnectivity-agent-c2ltb\" (UID: \"70679208-3538-46cc-8b71-0e27edb8344c\") " pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:18.473057 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.473028    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/950932a8-0a9d-4ba6-ba69-e9310263a09a-tmp\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.484515 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.484490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wplkk\" (UniqueName: \"kubernetes.io/projected/950932a8-0a9d-4ba6-ba69-e9310263a09a-kube-api-access-wplkk\") pod \"tuned-kbcz9\" (UID: \"950932a8-0a9d-4ba6-ba69-e9310263a09a\") " pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.485733 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.485710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh682\" (UniqueName: \"kubernetes.io/projected/b0c960b1-2e8e-410e-93c1-fbc4bcefc677-kube-api-access-nh682\") pod \"aws-ebs-csi-driver-node-jkdrh\" (UID: \"b0c960b1-2e8e-410e-93c1-fbc4bcefc677\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.486058 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.486032    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh74k\" (UniqueName: \"kubernetes.io/projected/0ed56e16-95f6-4ab2-be2d-2e1798e82604-kube-api-access-jh74k\") pod \"multus-additional-cni-plugins-5jnqf\" (UID: \"0ed56e16-95f6-4ab2-be2d-2e1798e82604\") " pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.486193 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.486170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvkxh\" (UniqueName: \"kubernetes.io/projected/b848d323-22e3-44fa-a82b-2994f66c296a-kube-api-access-kvkxh\") pod \"node-ca-vfgrc\" (UID: \"b848d323-22e3-44fa-a82b-2994f66c296a\") " pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.486426 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.486408    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh67x\" (UniqueName: \"kubernetes.io/projected/95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2-kube-api-access-vh67x\") pod \"multus-7h88j\" (UID: \"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2\") " pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.486795 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.486761    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn6l2\" (UniqueName: \"kubernetes.io/projected/f63d5f94-73d9-4236-9ed5-3f6fa4da022a-kube-api-access-wn6l2\") pod \"iptables-alerter-d4dlj\" (UID: \"f63d5f94-73d9-4236-9ed5-3f6fa4da022a\") " pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.563880 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.563845    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-75gck"
Jul 21 12:11:18.575804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.575761    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:18.584529 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.584510    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:18.589159 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.589140    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kbcz9"
Jul 21 12:11:18.596665 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.596648    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7h88j"
Jul 21 12:11:18.604308 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.604280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vfgrc"
Jul 21 12:11:18.612972 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.612957    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5jnqf"
Jul 21 12:11:18.619594 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.619576    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh"
Jul 21 12:11:18.626109 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.626091    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d4dlj"
Jul 21 12:11:18.874805 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.874721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:18.874805 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.874756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:18.875016 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.874874    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:18.875016 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.874887    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:11:18.875016 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.874909    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:11:18.875016 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.874921    2562 projected.go:196] Error preparing data for projected volume kube-api-access-78248 for pod openshift-network-diagnostics/network-check-target-lpnp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:18.875016 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.874944    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs podName:fa35c26d-86c4-456a-862f-fdc14069ec25 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:19.874922828 +0000 UTC m=+3.933353818 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs") pod "network-metrics-daemon-4tb74" (UID: "fa35c26d-86c4-456a-862f-fdc14069ec25") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:18.875016 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.874962    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248 podName:b788c24c-d635-4fcb-9f86-4ae4b2a1f56d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:19.874950744 +0000 UTC m=+3.933381725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-78248" (UniqueName: "kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248") pod "network-check-target-lpnp2" (UID: "b788c24c-d635-4fcb-9f86-4ae4b2a1f56d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:18.975572 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:18.975541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:18.975720 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.975684    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:18.975788 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:18.975744    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret podName:5b6c7079-8edc-469b-af1c-61f5ccfa0511 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:19.975725033 +0000 UTC m=+4.034156016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret") pod "global-pull-secret-syncer-q7tnn" (UID: "5b6c7079-8edc-469b-af1c-61f5ccfa0511") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:19.218163 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:19.218139    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb848d323_22e3_44fa_a82b_2994f66c296a.slice/crio-17da785ccd554e1de9864cd70e84875e0a21ddf3b9448b245efa0061bef6fc59 WatchSource:0}: Error finding container 17da785ccd554e1de9864cd70e84875e0a21ddf3b9448b245efa0061bef6fc59: Status 404 returned error can't find the container with id 17da785ccd554e1de9864cd70e84875e0a21ddf3b9448b245efa0061bef6fc59
Jul 21 12:11:19.220365 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:19.220333    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163cac4c_f48f_41af_9e32_faf67720471d.slice/crio-87a38b17fc0fbb6ecc5cfd81c288e7b4a079b98ed181742b069debbd47fd0bba WatchSource:0}: Error finding container 87a38b17fc0fbb6ecc5cfd81c288e7b4a079b98ed181742b069debbd47fd0bba: Status 404 returned error can't find the container with id 87a38b17fc0fbb6ecc5cfd81c288e7b4a079b98ed181742b069debbd47fd0bba
Jul 21 12:11:19.221890 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:19.221843    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed56e16_95f6_4ab2_be2d_2e1798e82604.slice/crio-c3b9e70073ecfd3275de311405e774afa64dfd986119a86ac245da3a4a340b7b WatchSource:0}: Error finding container c3b9e70073ecfd3275de311405e774afa64dfd986119a86ac245da3a4a340b7b: Status 404 returned error can't find the container with id c3b9e70073ecfd3275de311405e774afa64dfd986119a86ac245da3a4a340b7b
Jul 21 12:11:19.222123 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:19.222098    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a5e8dd_c9ca_48cf_ad4c_c149263ac9c2.slice/crio-0c311e331d986bf6226dcf8a6a01d22e14323f8b9b102b64b943016762726097 WatchSource:0}: Error finding container 0c311e331d986bf6226dcf8a6a01d22e14323f8b9b102b64b943016762726097: Status 404 returned error can't find the container with id 0c311e331d986bf6226dcf8a6a01d22e14323f8b9b102b64b943016762726097
Jul 21 12:11:19.225107 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:19.225084    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod950932a8_0a9d_4ba6_ba69_e9310263a09a.slice/crio-064f4fad53e3faad62e9fc992762bfb84491b5182e4feb318e9eb19ada637dcb WatchSource:0}: Error finding container 064f4fad53e3faad62e9fc992762bfb84491b5182e4feb318e9eb19ada637dcb: Status 404 returned error can't find the container with id 064f4fad53e3faad62e9fc992762bfb84491b5182e4feb318e9eb19ada637dcb
Jul 21 12:11:19.246411 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:19.246385    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c960b1_2e8e_410e_93c1_fbc4bcefc677.slice/crio-ba5caca02ed42ae211a0989c820d603e1574847553f1bb77bb4c48d0ae78d456 WatchSource:0}: Error finding container ba5caca02ed42ae211a0989c820d603e1574847553f1bb77bb4c48d0ae78d456: Status 404 returned error can't find the container with id ba5caca02ed42ae211a0989c820d603e1574847553f1bb77bb4c48d0ae78d456
Jul 21 12:11:19.247361 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:19.247337    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bc8242_dbb7_49de_a524_4c1ec938353b.slice/crio-b2a1424c3966cf8d09314b85fedce38894a259b7e62ab178e90e6331dfd861b9 WatchSource:0}: Error finding container b2a1424c3966cf8d09314b85fedce38894a259b7e62ab178e90e6331dfd861b9: Status 404 returned error can't find the container with id b2a1424c3966cf8d09314b85fedce38894a259b7e62ab178e90e6331dfd861b9
Jul 21 12:11:19.248561 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:19.248528    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf63d5f94_73d9_4236_9ed5_3f6fa4da022a.slice/crio-445ccc26b5bb1d2f4acb9f8d940ca702f04208dc5844b9b5dc0a02336448e141 WatchSource:0}: Error finding container 445ccc26b5bb1d2f4acb9f8d940ca702f04208dc5844b9b5dc0a02336448e141: Status 404 returned error can't find the container with id 445ccc26b5bb1d2f4acb9f8d940ca702f04208dc5844b9b5dc0a02336448e141
Jul 21 12:11:19.249344 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:19.249324    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70679208_3538_46cc_8b71_0e27edb8344c.slice/crio-7d984260b24cce69a11edf2924828ed02116193b2121ca65bafdb6369fb26e14 WatchSource:0}: Error finding container 7d984260b24cce69a11edf2924828ed02116193b2121ca65bafdb6369fb26e14: Status 404 returned error can't find the container with id 7d984260b24cce69a11edf2924828ed02116193b2121ca65bafdb6369fb26e14
Jul 21 12:11:19.304903 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.304756    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:06:18 +0000 UTC" deadline="2028-02-18 02:49:39.50306315 +0000 UTC"
Jul 21 12:11:19.304982 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.304905    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13838h38m20.198163432s"
Jul 21 12:11:19.420879 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.420847    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c2ltb" event={"ID":"70679208-3538-46cc-8b71-0e27edb8344c","Type":"ContainerStarted","Data":"7d984260b24cce69a11edf2924828ed02116193b2121ca65bafdb6369fb26e14"}
Jul 21 12:11:19.422034 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.422010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d4dlj" event={"ID":"f63d5f94-73d9-4236-9ed5-3f6fa4da022a","Type":"ContainerStarted","Data":"445ccc26b5bb1d2f4acb9f8d940ca702f04208dc5844b9b5dc0a02336448e141"}
Jul 21 12:11:19.423084 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.423061    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh" event={"ID":"b0c960b1-2e8e-410e-93c1-fbc4bcefc677","Type":"ContainerStarted","Data":"ba5caca02ed42ae211a0989c820d603e1574847553f1bb77bb4c48d0ae78d456"}
Jul 21 12:11:19.424207 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.424185    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kbcz9" event={"ID":"950932a8-0a9d-4ba6-ba69-e9310263a09a","Type":"ContainerStarted","Data":"064f4fad53e3faad62e9fc992762bfb84491b5182e4feb318e9eb19ada637dcb"}
Jul 21 12:11:19.428367 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.426374    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7h88j" event={"ID":"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2","Type":"ContainerStarted","Data":"0c311e331d986bf6226dcf8a6a01d22e14323f8b9b102b64b943016762726097"}
Jul 21 12:11:19.428367 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.427906    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vfgrc" event={"ID":"b848d323-22e3-44fa-a82b-2994f66c296a","Type":"ContainerStarted","Data":"17da785ccd554e1de9864cd70e84875e0a21ddf3b9448b245efa0061bef6fc59"}
Jul 21 12:11:19.430388 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.430363    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" event={"ID":"378731304c8222449ffad4ab0f72e94a","Type":"ContainerStarted","Data":"b13dceba4fc28dfc17c45320fce4e3f56221e17c936c7a01f291050dff2a1359"}
Jul 21 12:11:19.431455 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.431427    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" event={"ID":"24bc8242-dbb7-49de-a524-4c1ec938353b","Type":"ContainerStarted","Data":"b2a1424c3966cf8d09314b85fedce38894a259b7e62ab178e90e6331dfd861b9"}
Jul 21 12:11:19.432531 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.432511    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jnqf" event={"ID":"0ed56e16-95f6-4ab2-be2d-2e1798e82604","Type":"ContainerStarted","Data":"c3b9e70073ecfd3275de311405e774afa64dfd986119a86ac245da3a4a340b7b"}
Jul 21 12:11:19.433495 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.433476    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-75gck" event={"ID":"163cac4c-f48f-41af-9e32-faf67720471d","Type":"ContainerStarted","Data":"87a38b17fc0fbb6ecc5cfd81c288e7b4a079b98ed181742b069debbd47fd0bba"}
Jul 21 12:11:19.882537 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.882445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:19.882537 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.882505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:19.882750 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:19.882669    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:19.882750 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:19.882734    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs podName:fa35c26d-86c4-456a-862f-fdc14069ec25 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:21.882715049 +0000 UTC m=+5.941146056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs") pod "network-metrics-daemon-4tb74" (UID: "fa35c26d-86c4-456a-862f-fdc14069ec25") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:19.883420 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:19.883397    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:11:19.883516 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:19.883428    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:11:19.883516 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:19.883441    2562 projected.go:196] Error preparing data for projected volume kube-api-access-78248 for pod openshift-network-diagnostics/network-check-target-lpnp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:19.883516 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:19.883488    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248 podName:b788c24c-d635-4fcb-9f86-4ae4b2a1f56d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:21.883473559 +0000 UTC m=+5.941904556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-78248" (UniqueName: "kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248") pod "network-check-target-lpnp2" (UID: "b788c24c-d635-4fcb-9f86-4ae4b2a1f56d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:19.983033 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:19.982993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:19.983239 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:19.983219    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:19.983303 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:19.983293    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret podName:5b6c7079-8edc-469b-af1c-61f5ccfa0511 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:21.983273233 +0000 UTC m=+6.041704221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret") pod "global-pull-secret-syncer-q7tnn" (UID: "5b6c7079-8edc-469b-af1c-61f5ccfa0511") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:20.306000 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:20.305871    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:06:18 +0000 UTC" deadline="2028-02-10 03:57:46.05514433 +0000 UTC"
Jul 21 12:11:20.306000 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:20.305928    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13647h46m25.749219928s"
Jul 21 12:11:20.417481 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:20.417455    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:20.417645 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:20.417567    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:20.418091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:20.417983    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:20.418171 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:20.418089    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:20.418171 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:20.418150    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:20.418284 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:20.418222    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:20.456233 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:20.456198    2562 generic.go:356] "Generic (PLEG): container finished" podID="677db6ca0d35088a971975beecb717c8" containerID="6834c0232d751e787bf2722cb8a3f1899b487c102169a9b4791476063a79c605" exitCode=0
Jul 21 12:11:20.456892 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:20.456826    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" event={"ID":"677db6ca0d35088a971975beecb717c8","Type":"ContainerDied","Data":"6834c0232d751e787bf2722cb8a3f1899b487c102169a9b4791476063a79c605"}
Jul 21 12:11:20.478868 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:20.477256    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-237.ec2.internal" podStartSLOduration=3.477236855 podStartE2EDuration="3.477236855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:19.447839578 +0000 UTC m=+3.506270581" watchObservedRunningTime="2026-07-21 12:11:20.477236855 +0000 UTC m=+4.535667861"
Jul 21 12:11:21.471347 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:21.471311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" event={"ID":"677db6ca0d35088a971975beecb717c8","Type":"ContainerStarted","Data":"7a73fde63c75675b8dfb9e580e64e4f44d6b2e762f0529966aff8dff74eb03de"}
Jul 21 12:11:21.899429 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:21.898912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:21.899429 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:21.898962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:21.899429 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:21.899131    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:21.899429 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:21.899191    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs podName:fa35c26d-86c4-456a-862f-fdc14069ec25 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:25.899173006 +0000 UTC m=+9.957603995 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs") pod "network-metrics-daemon-4tb74" (UID: "fa35c26d-86c4-456a-862f-fdc14069ec25") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:21.899756 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:21.899590    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:11:21.899756 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:21.899607    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:11:21.899756 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:21.899620    2562 projected.go:196] Error preparing data for projected volume kube-api-access-78248 for pod openshift-network-diagnostics/network-check-target-lpnp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:21.899756 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:21.899659    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248 podName:b788c24c-d635-4fcb-9f86-4ae4b2a1f56d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:25.89964631 +0000 UTC m=+9.958077296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-78248" (UniqueName: "kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248") pod "network-check-target-lpnp2" (UID: "b788c24c-d635-4fcb-9f86-4ae4b2a1f56d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:21.999713 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:21.999677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:21.999923 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:21.999903    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:21.999993 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:21.999975    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret podName:5b6c7079-8edc-469b-af1c-61f5ccfa0511 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:25.99995557 +0000 UTC m=+10.058386575 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret") pod "global-pull-secret-syncer-q7tnn" (UID: "5b6c7079-8edc-469b-af1c-61f5ccfa0511") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:22.417679 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:22.416918    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:22.417679 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:22.417029    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:22.417679 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:22.417386    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:22.417679 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:22.417480    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:22.417679 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:22.417535    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:22.417679 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:22.417617    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:24.415380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:24.415349    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:24.415800 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:24.415453    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:24.415800 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:24.415765    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:24.415898 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:24.415860    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:24.415930 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:24.415913    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:24.415990 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:24.415955    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:25.933741 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:25.933702    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:25.934164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:25.933752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:25.934164 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:25.933900    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:25.934164 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:25.933965    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs podName:fa35c26d-86c4-456a-862f-fdc14069ec25 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:33.933945093 +0000 UTC m=+17.992376076 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs") pod "network-metrics-daemon-4tb74" (UID: "fa35c26d-86c4-456a-862f-fdc14069ec25") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:25.934399 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:25.934380    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:11:25.934474 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:25.934408    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:11:25.934474 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:25.934421    2562 projected.go:196] Error preparing data for projected volume kube-api-access-78248 for pod openshift-network-diagnostics/network-check-target-lpnp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:25.934474 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:25.934465    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248 podName:b788c24c-d635-4fcb-9f86-4ae4b2a1f56d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:33.934450581 +0000 UTC m=+17.992881577 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-78248" (UniqueName: "kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248") pod "network-check-target-lpnp2" (UID: "b788c24c-d635-4fcb-9f86-4ae4b2a1f56d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:26.034922 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:26.034323    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:26.034922 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:26.034506    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:26.034922 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:26.034566    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret podName:5b6c7079-8edc-469b-af1c-61f5ccfa0511 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:34.034548922 +0000 UTC m=+18.092979905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret") pod "global-pull-secret-syncer-q7tnn" (UID: "5b6c7079-8edc-469b-af1c-61f5ccfa0511") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:26.412507 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:26.412427    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:26.412656 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:26.412527    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:26.412841 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:26.412819    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:26.412951 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:26.412891    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:26.412951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:26.412925    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:26.413026 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:26.412964    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:28.411964 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:28.411896    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:28.412436 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:28.412038    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:28.412518 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:28.412454    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:28.412569 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:28.412548    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:28.414144 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:28.414114    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:28.414256 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:28.414200    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:30.411316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:30.411282    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:30.411723 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:30.411319    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:30.411723 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:30.411414    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:30.411854 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:30.411811    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:30.411914 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:30.411856    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:30.411950 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:30.411917    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:32.411079 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:32.411033    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:32.411079 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:32.411066    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:32.411555 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:32.411033    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:32.411555 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:32.411171    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:32.411555 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:32.411302    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:32.411555 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:32.411378    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:33.997420 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:33.997383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:33.997420 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:33.997420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:33.997865 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:33.997526    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:33.997865 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:33.997548    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:11:33.997865 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:33.997573    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:11:33.997865 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:33.997582    2562 projected.go:196] Error preparing data for projected volume kube-api-access-78248 for pod openshift-network-diagnostics/network-check-target-lpnp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:33.997865 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:33.997588    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs podName:fa35c26d-86c4-456a-862f-fdc14069ec25 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:49.997570305 +0000 UTC m=+34.056001288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs") pod "network-metrics-daemon-4tb74" (UID: "fa35c26d-86c4-456a-862f-fdc14069ec25") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:33.997865 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:33.997622    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248 podName:b788c24c-d635-4fcb-9f86-4ae4b2a1f56d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:49.997610873 +0000 UTC m=+34.056041858 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-78248" (UniqueName: "kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248") pod "network-check-target-lpnp2" (UID: "b788c24c-d635-4fcb-9f86-4ae4b2a1f56d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:34.097837 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:34.097800    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:34.097999 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:34.097959    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:34.098035 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:34.098029    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret podName:5b6c7079-8edc-469b-af1c-61f5ccfa0511 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:50.098014681 +0000 UTC m=+34.156445666 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret") pod "global-pull-secret-syncer-q7tnn" (UID: "5b6c7079-8edc-469b-af1c-61f5ccfa0511") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:34.411514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:34.411004    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:34.411514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:34.411010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:34.411514 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:34.411133    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:34.411514 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:34.411206    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:34.411514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:34.411258    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:34.411514 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:34.411331    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:36.412101 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:36.412063    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:36.412535 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:36.412150    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:36.412535 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:36.412189    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:36.412535 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:36.412214    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:36.412535 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:36.412274    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:36.412535 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:36.412368    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:38.412412 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:38.411926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:38.412412 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:38.411933    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:38.412412 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:38.412047    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:38.412412 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:38.412085    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:38.412412 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:38.412200    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:38.412412 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:38.412279    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:39.519204 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.518617    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-75gck" event={"ID":"163cac4c-f48f-41af-9e32-faf67720471d","Type":"ContainerStarted","Data":"9560dfab4058efe81ae4a4c2d132908694742bb2f2f23b2dcb732fda4eeef540"}
Jul 21 12:11:39.520405 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.520377    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c2ltb" event={"ID":"70679208-3538-46cc-8b71-0e27edb8344c","Type":"ContainerStarted","Data":"73fd2898608ef374712900acc575e629518828804e68c0c0d17397075bb7a620"}
Jul 21 12:11:39.522113 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.522087    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh" event={"ID":"b0c960b1-2e8e-410e-93c1-fbc4bcefc677","Type":"ContainerStarted","Data":"c71e409db23aff93f319499c4f2f08c539c24d4faeb8301e46bdc76a450996f4"}
Jul 21 12:11:39.523651 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.523626    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kbcz9" event={"ID":"950932a8-0a9d-4ba6-ba69-e9310263a09a","Type":"ContainerStarted","Data":"136a4d71063898eb6afc88c767368a87a008e8a6b27c3f5c027af8f1751f5e44"}
Jul 21 12:11:39.525300 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.525279    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7h88j" event={"ID":"95a5e8dd-c9ca-48cf-ad4c-c149263ac9c2","Type":"ContainerStarted","Data":"aedbceb8271be56bf3c909ec72245d378bd26c58e0a1ad61a4c00cc474b248de"}
Jul 21 12:11:39.526824 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.526803    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vfgrc" event={"ID":"b848d323-22e3-44fa-a82b-2994f66c296a","Type":"ContainerStarted","Data":"d6c85df81e2ea789c0ba478b5173a5d61625b545192e40d32cacf15fb83e0b89"}
Jul 21 12:11:39.530127 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.530101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" event={"ID":"24bc8242-dbb7-49de-a524-4c1ec938353b","Type":"ContainerStarted","Data":"d77b4bff53943407b55e87cd563792c70b7132e29f7763abf6bb3f7312e6e519"}
Jul 21 12:11:39.530231 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.530169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" event={"ID":"24bc8242-dbb7-49de-a524-4c1ec938353b","Type":"ContainerStarted","Data":"70cea391909562c7c9ca2a25c471654f7d911c707f09069ca4e142344a37c6da"}
Jul 21 12:11:39.530231 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.530218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" event={"ID":"24bc8242-dbb7-49de-a524-4c1ec938353b","Type":"ContainerStarted","Data":"a17df5d01109cf615f0fa104e3581e0d84c4f07f7f17d1869e8a1becabe30db4"}
Jul 21 12:11:39.530334 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.530263    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" event={"ID":"24bc8242-dbb7-49de-a524-4c1ec938353b","Type":"ContainerStarted","Data":"7d99f75585770055683ed4230a94d0e1ee31d820e6b890b88b47402358fe98d4"}
Jul 21 12:11:39.530334 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.530306    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" event={"ID":"24bc8242-dbb7-49de-a524-4c1ec938353b","Type":"ContainerStarted","Data":"f934c7ece74ea7f2261f002ca9fda0ef67340e201da1ba65636dd837fe6b73d8"}
Jul 21 12:11:39.530399 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.530357    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" event={"ID":"24bc8242-dbb7-49de-a524-4c1ec938353b","Type":"ContainerStarted","Data":"9fd4d284c00d2649fe4c9c0421d319d815308be59a93951b5d2fef2c63f7e8cb"}
Jul 21 12:11:39.531538 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.531515    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ed56e16-95f6-4ab2-be2d-2e1798e82604" containerID="d01868571ee52cc723f7dd6e70d0d72c641820f1e580f122619138d5ff2ec9e1" exitCode=0
Jul 21 12:11:39.531630 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.531550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jnqf" event={"ID":"0ed56e16-95f6-4ab2-be2d-2e1798e82604","Type":"ContainerDied","Data":"d01868571ee52cc723f7dd6e70d0d72c641820f1e580f122619138d5ff2ec9e1"}
Jul 21 12:11:39.534102 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.534065    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-237.ec2.internal" podStartSLOduration=22.534037885 podStartE2EDuration="22.534037885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:21.492006673 +0000 UTC m=+5.550437677" watchObservedRunningTime="2026-07-21 12:11:39.534037885 +0000 UTC m=+23.592468885"
Jul 21 12:11:39.534527 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.534498    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-75gck" podStartSLOduration=4.246218491 podStartE2EDuration="23.534489031s" totalImagesPullingTime="19.28827054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:39.534191496 +0000 UTC m=+23.592622501" watchObservedRunningTime="2026-07-21 12:11:39.534489031 +0000 UTC m=+23.592920035"
Jul 21 12:11:39.549919 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.549871    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c2ltb" podStartSLOduration=4.287229017 podStartE2EDuration="23.549853226s" totalImagesPullingTime="19.262624209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:39.549522074 +0000 UTC m=+23.607953077" watchObservedRunningTime="2026-07-21 12:11:39.549853226 +0000 UTC m=+23.608284233"
Jul 21 12:11:39.586493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.586440    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kbcz9" podStartSLOduration=4.320520293 podStartE2EDuration="23.586421952s" totalImagesPullingTime="19.265901659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:39.586398375 +0000 UTC m=+23.644829378" watchObservedRunningTime="2026-07-21 12:11:39.586421952 +0000 UTC m=+23.644852953"
Jul 21 12:11:39.602612 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.602557    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7h88j" podStartSLOduration=4.275771774 podStartE2EDuration="23.602539525s" totalImagesPullingTime="19.326767751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:39.602338672 +0000 UTC m=+23.660769675" watchObservedRunningTime="2026-07-21 12:11:39.602539525 +0000 UTC m=+23.660970528"
Jul 21 12:11:39.617204 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:39.617160    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vfgrc" podStartSLOduration=4.325733232 podStartE2EDuration="23.617146326s" totalImagesPullingTime="19.291413094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:39.617111268 +0000 UTC m=+23.675542273" watchObservedRunningTime="2026-07-21 12:11:39.617146326 +0000 UTC m=+23.675577389"
Jul 21 12:11:40.181164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.181136    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 12:11:40.327831 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.327716    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T12:11:40.181158223Z","UUID":"a68137d6-87be-4b77-a982-494dfd6d7ece","Handler":null,"Name":"","Endpoint":""}
Jul 21 12:11:40.329169 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.329149    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 12:11:40.329311 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.329189    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 12:11:40.411836 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.411738    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:40.412031 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:40.411900    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:40.412031 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.411740    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:40.412031 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.411738    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:40.412031 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:40.411993    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:40.412210 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:40.412092    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:40.535875 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.535838    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d4dlj" event={"ID":"f63d5f94-73d9-4236-9ed5-3f6fa4da022a","Type":"ContainerStarted","Data":"eaaa0f73853bfaad8adaad5565b9a677437f094241e5b3cf68639b8465121fd7"}
Jul 21 12:11:40.539012 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.538981    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh" event={"ID":"b0c960b1-2e8e-410e-93c1-fbc4bcefc677","Type":"ContainerStarted","Data":"6e8e4d7f63be0f89bfc64e9788a4d5f4bc38cf6281b325889b841216863eb31a"}
Jul 21 12:11:40.552721 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:40.552652    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d4dlj" podStartSLOduration=5.285066761 podStartE2EDuration="24.552633145s" totalImagesPullingTime="19.267566384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:40.552413538 +0000 UTC m=+24.610844545" watchObservedRunningTime="2026-07-21 12:11:40.552633145 +0000 UTC m=+24.611064151"
Jul 21 12:11:42.092027 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.091752    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:42.092488 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.092404    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:42.411756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.411673    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:42.411756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.411709    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:42.411756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.411709    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:42.412051 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:42.411814    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:42.412051 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:42.411923    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:42.412051 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:42.412009    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:42.545193 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.545152    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh" event={"ID":"b0c960b1-2e8e-410e-93c1-fbc4bcefc677","Type":"ContainerStarted","Data":"46470e73ba1ea6f7d8eb625bde23a7c32f86b00d35168111d67176752f589d1d"}
Jul 21 12:11:42.553001 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.552951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" event={"ID":"24bc8242-dbb7-49de-a524-4c1ec938353b","Type":"ContainerStarted","Data":"9393d3307876d4704f4c5ff32772ba3f603e7af8ad7ed541d7c4d31493a41c59"}
Jul 21 12:11:42.553195 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.553174    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:42.553677 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.553654    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c2ltb"
Jul 21 12:11:42.563728 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:42.563679    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkdrh" podStartSLOduration=4.657295684 podStartE2EDuration="26.563664985s" totalImagesPullingTime="21.906369301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:42.563281889 +0000 UTC m=+26.621712893" watchObservedRunningTime="2026-07-21 12:11:42.563664985 +0000 UTC m=+26.622095989"
Jul 21 12:11:44.411481 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:44.411309    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:44.412256 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:44.411321    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:44.412256 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:44.411562    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:44.412256 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:44.411370    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:44.412256 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:44.411638    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:44.412256 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:44.411699    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:44.560768 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:44.560731    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" event={"ID":"24bc8242-dbb7-49de-a524-4c1ec938353b","Type":"ContainerStarted","Data":"87947324af10f403f240dc0286f87819f5b2631e00a8896e9775050024bf2bee"}
Jul 21 12:11:44.561022 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:44.561005    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:44.562725 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:44.562696    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ed56e16-95f6-4ab2-be2d-2e1798e82604" containerID="70ad5dcf5eed37776a2146e5f64a75f416e0d7e2f43e9f7eb3a1e6169cf07b41" exitCode=0
Jul 21 12:11:44.562880 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:44.562766    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jnqf" event={"ID":"0ed56e16-95f6-4ab2-be2d-2e1798e82604","Type":"ContainerDied","Data":"70ad5dcf5eed37776a2146e5f64a75f416e0d7e2f43e9f7eb3a1e6169cf07b41"}
Jul 21 12:11:44.577216 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:44.577194    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:44.600586 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:44.600539    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw" podStartSLOduration=9.235794589 podStartE2EDuration="28.60052513s" totalImagesPullingTime="19.364730541s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:44.599871462 +0000 UTC m=+28.658302466" watchObservedRunningTime="2026-07-21 12:11:44.60052513 +0000 UTC m=+28.658956135"
Jul 21 12:11:45.565689 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:45.565649    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:45.566160 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:45.565791    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:45.582156 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:45.582125    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:11:46.374769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:46.374739    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lpnp2"]
Jul 21 12:11:46.374986 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:46.374882    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:46.375054 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:46.374989    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:46.377037 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:46.377012    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q7tnn"]
Jul 21 12:11:46.377165 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:46.377124    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:46.377255 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:46.377235    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:46.377717 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:46.377698    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4tb74"]
Jul 21 12:11:46.377818 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:46.377805    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:46.377902 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:46.377883    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:46.570124 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:46.570087    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ed56e16-95f6-4ab2-be2d-2e1798e82604" containerID="0e24bf635bc0a48a7332f22214b3b07c9dbca75d470019f5b405a8405ad84e76" exitCode=0
Jul 21 12:11:46.570497 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:46.570172    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jnqf" event={"ID":"0ed56e16-95f6-4ab2-be2d-2e1798e82604","Type":"ContainerDied","Data":"0e24bf635bc0a48a7332f22214b3b07c9dbca75d470019f5b405a8405ad84e76"}
Jul 21 12:11:48.411322 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:48.411119    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:48.411835 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:48.411119    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:48.411835 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:48.411419    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:48.411835 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:48.411495    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:48.411835 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:48.411151    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:48.411835 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:48.411589    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:48.579546 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:48.579508    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ed56e16-95f6-4ab2-be2d-2e1798e82604" containerID="b96fbdd1aed8a56670582c7edd5d069d33c3a881fa760a13d585c2fafcf0189e" exitCode=0
Jul 21 12:11:48.579696 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:48.579551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jnqf" event={"ID":"0ed56e16-95f6-4ab2-be2d-2e1798e82604","Type":"ContainerDied","Data":"b96fbdd1aed8a56670582c7edd5d069d33c3a881fa760a13d585c2fafcf0189e"}
Jul 21 12:11:50.022725 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:50.022694    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:50.022725 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:50.022731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:50.023169 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.022867    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:50.023169 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.022877    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:11:50.023169 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.022896    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:11:50.023169 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.022909    2562 projected.go:196] Error preparing data for projected volume kube-api-access-78248 for pod openshift-network-diagnostics/network-check-target-lpnp2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:50.023169 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.022932    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs podName:fa35c26d-86c4-456a-862f-fdc14069ec25 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:22.022913299 +0000 UTC m=+66.081344306 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs") pod "network-metrics-daemon-4tb74" (UID: "fa35c26d-86c4-456a-862f-fdc14069ec25") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:11:50.023169 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.022947    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248 podName:b788c24c-d635-4fcb-9f86-4ae4b2a1f56d nodeName:}" failed. No retries permitted until 2026-07-21 12:12:22.022937746 +0000 UTC m=+66.081368750 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-78248" (UniqueName: "kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248") pod "network-check-target-lpnp2" (UID: "b788c24c-d635-4fcb-9f86-4ae4b2a1f56d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:11:50.123858 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:50.123827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:50.124049 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.123982    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:50.124049 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.124049    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret podName:5b6c7079-8edc-469b-af1c-61f5ccfa0511 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:22.124032172 +0000 UTC m=+66.182463157 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret") pod "global-pull-secret-syncer-q7tnn" (UID: "5b6c7079-8edc-469b-af1c-61f5ccfa0511") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:11:50.411960 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:50.411915    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:50.412149 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:50.411916    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:50.412149 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.412049    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7tnn" podUID="5b6c7079-8edc-469b-af1c-61f5ccfa0511"
Jul 21 12:11:50.412149 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:50.412093    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:50.412266 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.412210    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tb74" podUID="fa35c26d-86c4-456a-862f-fdc14069ec25"
Jul 21 12:11:50.412329 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:50.412308    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpnp2" podUID="b788c24c-d635-4fcb-9f86-4ae4b2a1f56d"
Jul 21 12:11:51.222947 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.222906    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-237.ec2.internal" event="NodeReady"
Jul 21 12:11:51.223472 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.223062    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 12:11:51.273288 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.273198    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tb2sh"]
Jul 21 12:11:51.276844 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.276819    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rwmgb"]
Jul 21 12:11:51.276974 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.276934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tb2sh"
Jul 21 12:11:51.280766 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.279328    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 12:11:51.280766 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.279486    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6rff6\""
Jul 21 12:11:51.280766 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.279683    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 12:11:51.280766 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.279737    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 12:11:51.280766 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.280609    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.282623 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.282569    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 12:11:51.282872 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.282854    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 12:11:51.282974 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.282886    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fnnf7\""
Jul 21 12:11:51.285892 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.285874    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tb2sh"]
Jul 21 12:11:51.290875 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.290857    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rwmgb"]
Jul 21 12:11:51.434083 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.434039    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxr6r\" (UniqueName: \"kubernetes.io/projected/4b3185c6-d223-4671-a986-0858e36e5190-kube-api-access-jxr6r\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.434246 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.434202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce7c7b6d-ce2c-4026-83de-c59e7d3abae9-cert\") pod \"ingress-canary-tb2sh\" (UID: \"ce7c7b6d-ce2c-4026-83de-c59e7d3abae9\") " pod="openshift-ingress-canary/ingress-canary-tb2sh"
Jul 21 12:11:51.434312 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.434246    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.434312 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.434277    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b3185c6-d223-4671-a986-0858e36e5190-tmp-dir\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.434312 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.434300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3185c6-d223-4671-a986-0858e36e5190-config-volume\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.434429 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.434316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2fjz\" (UniqueName: \"kubernetes.io/projected/ce7c7b6d-ce2c-4026-83de-c59e7d3abae9-kube-api-access-r2fjz\") pod \"ingress-canary-tb2sh\" (UID: \"ce7c7b6d-ce2c-4026-83de-c59e7d3abae9\") " pod="openshift-ingress-canary/ingress-canary-tb2sh"
Jul 21 12:11:51.535429 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.535337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b3185c6-d223-4671-a986-0858e36e5190-tmp-dir\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.535429 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.535380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3185c6-d223-4671-a986-0858e36e5190-config-volume\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.535429 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.535399    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2fjz\" (UniqueName: \"kubernetes.io/projected/ce7c7b6d-ce2c-4026-83de-c59e7d3abae9-kube-api-access-r2fjz\") pod \"ingress-canary-tb2sh\" (UID: \"ce7c7b6d-ce2c-4026-83de-c59e7d3abae9\") " pod="openshift-ingress-canary/ingress-canary-tb2sh"
Jul 21 12:11:51.535691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.535439    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxr6r\" (UniqueName: \"kubernetes.io/projected/4b3185c6-d223-4671-a986-0858e36e5190-kube-api-access-jxr6r\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.535691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.535514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce7c7b6d-ce2c-4026-83de-c59e7d3abae9-cert\") pod \"ingress-canary-tb2sh\" (UID: \"ce7c7b6d-ce2c-4026-83de-c59e7d3abae9\") " pod="openshift-ingress-canary/ingress-canary-tb2sh"
Jul 21 12:11:51.535691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.535540    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.535691 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:51.535643    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:11:51.535909 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:51.535704    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls podName:4b3185c6-d223-4671-a986-0858e36e5190 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:52.035684944 +0000 UTC m=+36.094115943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls") pod "dns-default-rwmgb" (UID: "4b3185c6-d223-4671-a986-0858e36e5190") : secret "dns-default-metrics-tls" not found
Jul 21 12:11:51.535909 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.535705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b3185c6-d223-4671-a986-0858e36e5190-tmp-dir\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.536072 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.535984    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3185c6-d223-4671-a986-0858e36e5190-config-volume\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.540539 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.540514    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce7c7b6d-ce2c-4026-83de-c59e7d3abae9-cert\") pod \"ingress-canary-tb2sh\" (UID: \"ce7c7b6d-ce2c-4026-83de-c59e7d3abae9\") " pod="openshift-ingress-canary/ingress-canary-tb2sh"
Jul 21 12:11:51.544854 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.544830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxr6r\" (UniqueName: \"kubernetes.io/projected/4b3185c6-d223-4671-a986-0858e36e5190-kube-api-access-jxr6r\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:51.545300 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.545277    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2fjz\" (UniqueName: \"kubernetes.io/projected/ce7c7b6d-ce2c-4026-83de-c59e7d3abae9-kube-api-access-r2fjz\") pod \"ingress-canary-tb2sh\" (UID: \"ce7c7b6d-ce2c-4026-83de-c59e7d3abae9\") " pod="openshift-ingress-canary/ingress-canary-tb2sh"
Jul 21 12:11:51.593794 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:51.593746    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tb2sh"
Jul 21 12:11:52.040686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.040645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:52.040901 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:52.040800    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:11:52.040901 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:52.040876    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls podName:4b3185c6-d223-4671-a986-0858e36e5190 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:53.040861541 +0000 UTC m=+37.099292524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls") pod "dns-default-rwmgb" (UID: "4b3185c6-d223-4671-a986-0858e36e5190") : secret "dns-default-metrics-tls" not found
Jul 21 12:11:52.411924 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.411891    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:11:52.412675 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.411890    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:11:52.412675 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.412094    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:11:52.415151 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.414759    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 12:11:52.415151 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.414825    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 12:11:52.416020 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.415705    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8v447\""
Jul 21 12:11:52.416020 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.415743    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2w9zn\""
Jul 21 12:11:52.416020 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.415762    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 12:11:52.416020 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.415808    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 12:11:52.937535 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:52.937385    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tb2sh"]
Jul 21 12:11:52.942098 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:52.942045    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7c7b6d_ce2c_4026_83de_c59e7d3abae9.slice/crio-2ed6ae051205866ac44c2d3293c0f41e11e1df6f4f869b64f262645c6c030b36 WatchSource:0}: Error finding container 2ed6ae051205866ac44c2d3293c0f41e11e1df6f4f869b64f262645c6c030b36: Status 404 returned error can't find the container with id 2ed6ae051205866ac44c2d3293c0f41e11e1df6f4f869b64f262645c6c030b36
Jul 21 12:11:53.049024 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:53.048990    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:53.049160 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:53.049142    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:11:53.049215 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:53.049206    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls podName:4b3185c6-d223-4671-a986-0858e36e5190 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:55.04919058 +0000 UTC m=+39.107621562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls") pod "dns-default-rwmgb" (UID: "4b3185c6-d223-4671-a986-0858e36e5190") : secret "dns-default-metrics-tls" not found
Jul 21 12:11:53.593488 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:53.593458    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ed56e16-95f6-4ab2-be2d-2e1798e82604" containerID="5fa23d40ac0743fb1e8a73b89890479081861640b81e1935944bbde6bc006080" exitCode=0
Jul 21 12:11:53.593914 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:53.593552    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jnqf" event={"ID":"0ed56e16-95f6-4ab2-be2d-2e1798e82604","Type":"ContainerDied","Data":"5fa23d40ac0743fb1e8a73b89890479081861640b81e1935944bbde6bc006080"}
Jul 21 12:11:53.594686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:53.594668    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tb2sh" event={"ID":"ce7c7b6d-ce2c-4026-83de-c59e7d3abae9","Type":"ContainerStarted","Data":"2ed6ae051205866ac44c2d3293c0f41e11e1df6f4f869b64f262645c6c030b36"}
Jul 21 12:11:54.601611 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.601580    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ed56e16-95f6-4ab2-be2d-2e1798e82604" containerID="4ecba4923aea2adaf716fe56996ab047456c9b56cdf9286fcd206f2c173870b8" exitCode=0
Jul 21 12:11:54.602037 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.601630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jnqf" event={"ID":"0ed56e16-95f6-4ab2-be2d-2e1798e82604","Type":"ContainerDied","Data":"4ecba4923aea2adaf716fe56996ab047456c9b56cdf9286fcd206f2c173870b8"}
Jul 21 12:11:54.817163 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.816977    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mdb7w"]
Jul 21 12:11:54.820276 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.820253    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:54.822445 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.822424    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 12:11:54.822588 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.822427    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 12:11:54.822644 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.822621    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2gbfg\""
Jul 21 12:11:54.822735 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.822718    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 12:11:54.822854 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.822838    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 12:11:54.830954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.830927    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mdb7w"]
Jul 21 12:11:54.965187 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.965094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-data-volume\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:54.965187 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.965139    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-crio-socket\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:54.965187 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.965178    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:54.965416 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.965229    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:54.965416 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:54.965273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6hb2\" (UniqueName: \"kubernetes.io/projected/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-api-access-x6hb2\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.065989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.065957    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-crio-socket\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.066164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.066014    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.066164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.066076    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.066164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.066113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hb2\" (UniqueName: \"kubernetes.io/projected/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-api-access-x6hb2\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.066340 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.066190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:55.066340 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.066197    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:11:55.066340 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.066231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-data-volume\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.066340 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.066268    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-crio-socket\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.066340 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.066283    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm podName:7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee nodeName:}" failed. No retries permitted until 2026-07-21 12:11:55.566259965 +0000 UTC m=+39.624690949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdb7w" (UID: "7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee") : configmap "kube-rbac-proxy" not found
Jul 21 12:11:55.066550 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.066369    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:11:55.066550 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.066416    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls podName:4b3185c6-d223-4671-a986-0858e36e5190 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:59.066400675 +0000 UTC m=+43.124831663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls") pod "dns-default-rwmgb" (UID: "4b3185c6-d223-4671-a986-0858e36e5190") : secret "dns-default-metrics-tls" not found
Jul 21 12:11:55.066629 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.066563    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:11:55.066629 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.066620    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls podName:7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee nodeName:}" failed. No retries permitted until 2026-07-21 12:11:55.566604516 +0000 UTC m=+39.625035502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdb7w" (UID: "7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:11:55.066890 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.066865    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-data-volume\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.085026 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.084992    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6hb2\" (UniqueName: \"kubernetes.io/projected/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-api-access-x6hb2\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.131392 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.131358    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv"]
Jul 21 12:11:55.135216 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.135193    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv"
Jul 21 12:11:55.137580 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.137556    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9gzn6\""
Jul 21 12:11:55.137697 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.137622    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 21 12:11:55.137809 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.137764    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 21 12:11:55.142968 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.142942    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv"]
Jul 21 12:11:55.267722 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.267642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mh2d\" (UniqueName: \"kubernetes.io/projected/c42d6ab4-ba5a-4b34-bd34-82652798ff36-kube-api-access-2mh2d\") pod \"migrator-7cc9c7ffd7-k9qlv\" (UID: \"c42d6ab4-ba5a-4b34-bd34-82652798ff36\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv"
Jul 21 12:11:55.368686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.368646    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mh2d\" (UniqueName: \"kubernetes.io/projected/c42d6ab4-ba5a-4b34-bd34-82652798ff36-kube-api-access-2mh2d\") pod \"migrator-7cc9c7ffd7-k9qlv\" (UID: \"c42d6ab4-ba5a-4b34-bd34-82652798ff36\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv"
Jul 21 12:11:55.376329 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.376304    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mh2d\" (UniqueName: \"kubernetes.io/projected/c42d6ab4-ba5a-4b34-bd34-82652798ff36-kube-api-access-2mh2d\") pod \"migrator-7cc9c7ffd7-k9qlv\" (UID: \"c42d6ab4-ba5a-4b34-bd34-82652798ff36\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv"
Jul 21 12:11:55.448068 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.448035    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv"
Jul 21 12:11:55.570982 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.570941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.571116 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.570997    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:55.571116 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.571082    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:11:55.571174 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.571133    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:11:55.571174 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.571165    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm podName:7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee nodeName:}" failed. No retries permitted until 2026-07-21 12:11:56.571143457 +0000 UTC m=+40.629574441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdb7w" (UID: "7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee") : configmap "kube-rbac-proxy" not found
Jul 21 12:11:55.571254 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:55.571194    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls podName:7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee nodeName:}" failed. No retries permitted until 2026-07-21 12:11:56.571175461 +0000 UTC m=+40.629606526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdb7w" (UID: "7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:11:55.594434 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.594403    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv"]
Jul 21 12:11:55.607659 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.607629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jnqf" event={"ID":"0ed56e16-95f6-4ab2-be2d-2e1798e82604","Type":"ContainerStarted","Data":"63c08feb2f5cf4ac655a9fb3657c137e5a954b2d772696db32fb2bd972bc650d"}
Jul 21 12:11:55.630914 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:55.630848    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5jnqf" podStartSLOduration=9.226021505 podStartE2EDuration="39.630827713s" totalImagesPullingTime="30.404806208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:55.629174319 +0000 UTC m=+39.687605322" watchObservedRunningTime="2026-07-21 12:11:55.630827713 +0000 UTC m=+39.689258718"
Jul 21 12:11:56.360309 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:56.360268    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42d6ab4_ba5a_4b34_bd34_82652798ff36.slice/crio-f66f49f28f3372f1a11a5d52da1434640d02e6bb12333ed36be4a7cea805534e WatchSource:0}: Error finding container f66f49f28f3372f1a11a5d52da1434640d02e6bb12333ed36be4a7cea805534e: Status 404 returned error can't find the container with id f66f49f28f3372f1a11a5d52da1434640d02e6bb12333ed36be4a7cea805534e
Jul 21 12:11:56.580585 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:56.580560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:56.580688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:56.580594    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:56.580750 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:56.580702    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:11:56.580750 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:56.580729    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:11:56.580848 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:56.580798    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm podName:7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.580756035 +0000 UTC m=+42.639187021 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdb7w" (UID: "7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee") : configmap "kube-rbac-proxy" not found
Jul 21 12:11:56.580848 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:56.580818    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls podName:7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.580808294 +0000 UTC m=+42.639239284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdb7w" (UID: "7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:11:56.604857 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:56.604837    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-75gck_163cac4c-f48f-41af-9e32-faf67720471d/dns-node-resolver/0.log"
Jul 21 12:11:56.610599 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:56.610551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv" event={"ID":"c42d6ab4-ba5a-4b34-bd34-82652798ff36","Type":"ContainerStarted","Data":"f66f49f28f3372f1a11a5d52da1434640d02e6bb12333ed36be4a7cea805534e"}
Jul 21 12:11:57.553076 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.552186    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-n9f55"]
Jul 21 12:11:57.575968 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.575925    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-n9f55"]
Jul 21 12:11:57.575968 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.575949    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.578463 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.578435    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 21 12:11:57.578463 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.578456    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 21 12:11:57.579451 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.579432    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 21 12:11:57.579616 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.579463    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-llf72\""
Jul 21 12:11:57.579616 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.579464    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 21 12:11:57.614814 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.614759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tb2sh" event={"ID":"ce7c7b6d-ce2c-4026-83de-c59e7d3abae9","Type":"ContainerStarted","Data":"563d4bf085ad54777624a1976404548434819abbbc35b9447c176b62abff022f"}
Jul 21 12:11:57.635714 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.635655    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tb2sh" podStartSLOduration=3.041955674 podStartE2EDuration="6.635634186s" totalImagesPullingTime="3.593678512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:57.634837401 +0000 UTC m=+41.693268405" watchObservedRunningTime="2026-07-21 12:11:57.635634186 +0000 UTC m=+41.694065188"
Jul 21 12:11:57.690156 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.690119    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/72e00e96-94b7-4c27-a536-31cfede909b4-signing-key\") pod \"service-ca-58598695f7-n9f55\" (UID: \"72e00e96-94b7-4c27-a536-31cfede909b4\") " pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.690404 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.690166    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56cw6\" (UniqueName: \"kubernetes.io/projected/72e00e96-94b7-4c27-a536-31cfede909b4-kube-api-access-56cw6\") pod \"service-ca-58598695f7-n9f55\" (UID: \"72e00e96-94b7-4c27-a536-31cfede909b4\") " pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.690404 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.690310    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/72e00e96-94b7-4c27-a536-31cfede909b4-signing-cabundle\") pod \"service-ca-58598695f7-n9f55\" (UID: \"72e00e96-94b7-4c27-a536-31cfede909b4\") " pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.791683 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.791641    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/72e00e96-94b7-4c27-a536-31cfede909b4-signing-cabundle\") pod \"service-ca-58598695f7-n9f55\" (UID: \"72e00e96-94b7-4c27-a536-31cfede909b4\") " pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.791880 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.791846    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/72e00e96-94b7-4c27-a536-31cfede909b4-signing-key\") pod \"service-ca-58598695f7-n9f55\" (UID: \"72e00e96-94b7-4c27-a536-31cfede909b4\") " pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.791956 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.791895    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56cw6\" (UniqueName: \"kubernetes.io/projected/72e00e96-94b7-4c27-a536-31cfede909b4-kube-api-access-56cw6\") pod \"service-ca-58598695f7-n9f55\" (UID: \"72e00e96-94b7-4c27-a536-31cfede909b4\") " pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.792391 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.792360    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/72e00e96-94b7-4c27-a536-31cfede909b4-signing-cabundle\") pod \"service-ca-58598695f7-n9f55\" (UID: \"72e00e96-94b7-4c27-a536-31cfede909b4\") " pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.794343 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.794314    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/72e00e96-94b7-4c27-a536-31cfede909b4-signing-key\") pod \"service-ca-58598695f7-n9f55\" (UID: \"72e00e96-94b7-4c27-a536-31cfede909b4\") " pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.799761 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.799735    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56cw6\" (UniqueName: \"kubernetes.io/projected/72e00e96-94b7-4c27-a536-31cfede909b4-kube-api-access-56cw6\") pod \"service-ca-58598695f7-n9f55\" (UID: \"72e00e96-94b7-4c27-a536-31cfede909b4\") " pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:57.805162 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.805119    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vfgrc_b848d323-22e3-44fa-a82b-2994f66c296a/node-ca/0.log"
Jul 21 12:11:57.886704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:57.886665    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-n9f55"
Jul 21 12:11:58.170913 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:58.170882    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-n9f55"]
Jul 21 12:11:58.173948 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:11:58.173911    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e00e96_94b7_4c27_a536_31cfede909b4.slice/crio-a8c821a46d341f9b9f20847ec8f687822c31b4b5c6ec6f39984a55b9836ace88 WatchSource:0}: Error finding container a8c821a46d341f9b9f20847ec8f687822c31b4b5c6ec6f39984a55b9836ace88: Status 404 returned error can't find the container with id a8c821a46d341f9b9f20847ec8f687822c31b4b5c6ec6f39984a55b9836ace88
Jul 21 12:11:58.599809 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:58.599706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:58.599809 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:58.599748    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:11:58.600039 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:58.599861    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:11:58.600039 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:58.599928    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm podName:7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee nodeName:}" failed. No retries permitted until 2026-07-21 12:12:02.59991334 +0000 UTC m=+46.658344325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdb7w" (UID: "7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee") : configmap "kube-rbac-proxy" not found
Jul 21 12:11:58.600039 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:58.599931    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:11:58.600039 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:58.599973    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls podName:7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee nodeName:}" failed. No retries permitted until 2026-07-21 12:12:02.599961157 +0000 UTC m=+46.658392139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdb7w" (UID: "7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:11:58.619381 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:58.619346    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv" event={"ID":"c42d6ab4-ba5a-4b34-bd34-82652798ff36","Type":"ContainerStarted","Data":"e61069d513888a99e11fa72eef81e7aa81413615c6d40eeb98ea82bc6fdbcab3"}
Jul 21 12:11:58.619816 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:58.619415    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv" event={"ID":"c42d6ab4-ba5a-4b34-bd34-82652798ff36","Type":"ContainerStarted","Data":"e0bb8e990a10ebc01ded54c1bc8929a6bb2bcf2e8dd160f119be03d04929fca2"}
Jul 21 12:11:58.620544 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:58.620518    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-n9f55" event={"ID":"72e00e96-94b7-4c27-a536-31cfede909b4","Type":"ContainerStarted","Data":"a8c821a46d341f9b9f20847ec8f687822c31b4b5c6ec6f39984a55b9836ace88"}
Jul 21 12:11:58.636029 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:58.635764    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-k9qlv" podStartSLOduration=1.9437556790000001 podStartE2EDuration="3.635748437s" totalImagesPullingTime="1.691992758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:58.635655573 +0000 UTC m=+42.694086571" watchObservedRunningTime="2026-07-21 12:11:58.635748437 +0000 UTC m=+42.694179443"
Jul 21 12:11:58.805609 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:58.805581    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tb2sh_ce7c7b6d-ce2c-4026-83de-c59e7d3abae9/serve-healthcheck-canary/0.log"
Jul 21 12:11:59.105172 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:11:59.105132    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:11:59.105325 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:59.105297    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:11:59.105377 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:11:59.105365    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls podName:4b3185c6-d223-4671-a986-0858e36e5190 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:07.105346248 +0000 UTC m=+51.163777235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls") pod "dns-default-rwmgb" (UID: "4b3185c6-d223-4671-a986-0858e36e5190") : secret "dns-default-metrics-tls" not found
Jul 21 12:12:01.630798 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:01.630747    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-n9f55" event={"ID":"72e00e96-94b7-4c27-a536-31cfede909b4","Type":"ContainerStarted","Data":"57e25dde3c6997cae94050119ad6b3f5d004314a1c3cacab34060daba645140c"}
Jul 21 12:12:01.670537 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:01.670467    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-n9f55" podStartSLOduration=2.218857417 podStartE2EDuration="4.670446339s" totalImagesPullingTime="2.451588922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:01.66887658 +0000 UTC m=+45.727307584" watchObservedRunningTime="2026-07-21 12:12:01.670446339 +0000 UTC m=+45.728877346"
Jul 21 12:12:02.634012 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:02.633941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:12:02.634474 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:02.634081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:12:02.634474 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:02.634163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:12:02.634474 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:12:02.634407    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:12:02.634474 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:12:02.634470    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls podName:7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee nodeName:}" failed. No retries permitted until 2026-07-21 12:12:10.634453411 +0000 UTC m=+54.692884406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdb7w" (UID: "7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:12:07.170551 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:07.170511    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:12:07.173117 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:07.173088    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3185c6-d223-4671-a986-0858e36e5190-metrics-tls\") pod \"dns-default-rwmgb\" (UID: \"4b3185c6-d223-4671-a986-0858e36e5190\") " pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:12:07.197990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:07.197951    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:12:07.356315 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:07.356283    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rwmgb"]
Jul 21 12:12:07.359614 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:07.359584    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3185c6_d223_4671_a986_0858e36e5190.slice/crio-adaea7cf51aa117a9b5c9efff6b79d12a8819b5feb23184d2885903f322f3379 WatchSource:0}: Error finding container adaea7cf51aa117a9b5c9efff6b79d12a8819b5feb23184d2885903f322f3379: Status 404 returned error can't find the container with id adaea7cf51aa117a9b5c9efff6b79d12a8819b5feb23184d2885903f322f3379
Jul 21 12:12:07.647482 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:07.647447    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rwmgb" event={"ID":"4b3185c6-d223-4671-a986-0858e36e5190","Type":"ContainerStarted","Data":"adaea7cf51aa117a9b5c9efff6b79d12a8819b5feb23184d2885903f322f3379"}
Jul 21 12:12:10.693578 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:10.693552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:12:10.696342 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:10.696305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdb7w\" (UID: \"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee\") " pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:12:10.730447 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:10.730417    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mdb7w"
Jul 21 12:12:10.909348 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:10.909311    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mdb7w"]
Jul 21 12:12:10.913643 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:10.913620    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b9466a7_98d1_467c_9ce7_5b69fdf5b0ee.slice/crio-0aef2d3b4db9e62ad2b02d9e5aa1ce3f2e7f419961cd99a1e74d6de2db12cfc1 WatchSource:0}: Error finding container 0aef2d3b4db9e62ad2b02d9e5aa1ce3f2e7f419961cd99a1e74d6de2db12cfc1: Status 404 returned error can't find the container with id 0aef2d3b4db9e62ad2b02d9e5aa1ce3f2e7f419961cd99a1e74d6de2db12cfc1
Jul 21 12:12:11.682060 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:11.681128    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rwmgb" event={"ID":"4b3185c6-d223-4671-a986-0858e36e5190","Type":"ContainerStarted","Data":"3589f05197d9337baf244c3119d4d3371096ceb025ca4dfa460dd534cc6561e5"}
Jul 21 12:12:11.682060 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:11.681231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rwmgb" event={"ID":"4b3185c6-d223-4671-a986-0858e36e5190","Type":"ContainerStarted","Data":"57892496c7d526d98195ecbff55084777f3391994761b5bbc56d79dd1419cb0b"}
Jul 21 12:12:11.682060 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:11.681992    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:12:11.683944 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:11.683791    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdb7w" event={"ID":"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee","Type":"ContainerStarted","Data":"0aef2d3b4db9e62ad2b02d9e5aa1ce3f2e7f419961cd99a1e74d6de2db12cfc1"}
Jul 21 12:12:11.703390 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:11.703324    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rwmgb" podStartSLOduration=17.432729049 podStartE2EDuration="20.70330367s" totalImagesPullingTime="3.270574621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:11.70058025 +0000 UTC m=+55.759011256" watchObservedRunningTime="2026-07-21 12:12:11.70330367 +0000 UTC m=+55.761734675"
Jul 21 12:12:16.704808 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:16.704704    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdb7w" event={"ID":"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee","Type":"ContainerStarted","Data":"e24fe6bb1cea9f75b08abc540e8cf60c742942d96285e5843002b15f2480c8d9"}
Jul 21 12:12:17.589103 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.589062    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbwbw"
Jul 21 12:12:17.800829 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.800790    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"]
Jul 21 12:12:17.876838 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.876750    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"]
Jul 21 12:12:17.876999 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.876897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:17.879517 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.879485    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 21 12:12:17.879765 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.879485    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 21 12:12:17.879765 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.879609    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 21 12:12:17.879938 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.879885    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 21 12:12:17.879999 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.879962    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 21 12:12:17.880057 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.879885    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 21 12:12:17.880438 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.880416    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 21 12:12:17.947114 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.947075    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:17.947290 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.947140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:17.947290 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.947201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-hub\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:17.947290 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.947266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88lcd\" (UniqueName: \"kubernetes.io/projected/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-kube-api-access-88lcd\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:17.947439 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.947334    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-ca\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:17.947439 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:17.947358    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.048540 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.048501    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.048735 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.048558    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.048735 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.048594    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-hub\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.048735 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.048639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88lcd\" (UniqueName: \"kubernetes.io/projected/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-kube-api-access-88lcd\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.048928 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.048819    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-ca\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.048928 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.048859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.049493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.049447    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.051566 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.051540    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-hub\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.051897 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.051878    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-ca\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.052086 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.052068    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.053012 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.052991    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.060481 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.060455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88lcd\" (UniqueName: \"kubernetes.io/projected/f6b7174f-ac0b-4698-b020-fdb8ea0913cb-kube-api-access-88lcd\") pod \"cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7\" (UID: \"f6b7174f-ac0b-4698-b020-fdb8ea0913cb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:18.193080 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:18.192992    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"
Jul 21 12:12:19.358093 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:19.358070    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7"]
Jul 21 12:12:19.361551 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:19.361522    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6b7174f_ac0b_4698_b020_fdb8ea0913cb.slice/crio-be3b08a002da166f91f72eb884d40a92f8a03c2d0d579df02fe8e98a584aaf45 WatchSource:0}: Error finding container be3b08a002da166f91f72eb884d40a92f8a03c2d0d579df02fe8e98a584aaf45: Status 404 returned error can't find the container with id be3b08a002da166f91f72eb884d40a92f8a03c2d0d579df02fe8e98a584aaf45
Jul 21 12:12:19.716945 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:19.716910    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7" event={"ID":"f6b7174f-ac0b-4698-b020-fdb8ea0913cb","Type":"ContainerStarted","Data":"be3b08a002da166f91f72eb884d40a92f8a03c2d0d579df02fe8e98a584aaf45"}
Jul 21 12:12:19.718535 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:19.718510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdb7w" event={"ID":"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee","Type":"ContainerStarted","Data":"679bdcbb890d119a2a8e8b8f0e4a27378ddf73203b81abde8f8bf89b293214a8"}
Jul 21 12:12:22.113078 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.113033    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:12:22.113486 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.113095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:12:22.115639 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.115615    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 12:12:22.115789 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.115698    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 12:12:22.126303 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.126106    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 12:12:22.126303 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.126250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa35c26d-86c4-456a-862f-fdc14069ec25-metrics-certs\") pod \"network-metrics-daemon-4tb74\" (UID: \"fa35c26d-86c4-456a-862f-fdc14069ec25\") " pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:12:22.137184 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.137158    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78248\" (UniqueName: \"kubernetes.io/projected/b788c24c-d635-4fcb-9f86-4ae4b2a1f56d-kube-api-access-78248\") pod \"network-check-target-lpnp2\" (UID: \"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d\") " pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:12:22.213859 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.213824    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:12:22.216634 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.216611    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 12:12:22.226554 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.226531    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5b6c7079-8edc-469b-af1c-61f5ccfa0511-original-pull-secret\") pod \"global-pull-secret-syncer-q7tnn\" (UID: \"5b6c7079-8edc-469b-af1c-61f5ccfa0511\") " pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:12:22.427173 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.427098    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2w9zn\""
Jul 21 12:12:22.434468 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.434444    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8v447\""
Jul 21 12:12:22.434605 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.434549    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tb74"
Jul 21 12:12:22.438695 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.438677    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7tnn"
Jul 21 12:12:22.442840 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.442813    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:12:22.695107 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:22.695028    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rwmgb"
Jul 21 12:12:25.251701 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.251673    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lpnp2"]
Jul 21 12:12:25.256824 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:25.256794    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb788c24c_d635_4fcb_9f86_4ae4b2a1f56d.slice/crio-65635c127bbc9b77646618d1f1cf173330632175988293e571c3f764e94fbaf1 WatchSource:0}: Error finding container 65635c127bbc9b77646618d1f1cf173330632175988293e571c3f764e94fbaf1: Status 404 returned error can't find the container with id 65635c127bbc9b77646618d1f1cf173330632175988293e571c3f764e94fbaf1
Jul 21 12:12:25.271449 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.271374    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4tb74"]
Jul 21 12:12:25.274920 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:25.274892    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa35c26d_86c4_456a_862f_fdc14069ec25.slice/crio-840c081b28bafea9e7451fd50d4b3a3fd435eb4b21ee9565de8c77c174ab2a67 WatchSource:0}: Error finding container 840c081b28bafea9e7451fd50d4b3a3fd435eb4b21ee9565de8c77c174ab2a67: Status 404 returned error can't find the container with id 840c081b28bafea9e7451fd50d4b3a3fd435eb4b21ee9565de8c77c174ab2a67
Jul 21 12:12:25.287643 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.287620    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q7tnn"]
Jul 21 12:12:25.290444 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:25.290420    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b6c7079_8edc_469b_af1c_61f5ccfa0511.slice/crio-a4e0bfd6fa6e812e8bdd4b4f17349196632bff20736035475898eed432f5abec WatchSource:0}: Error finding container a4e0bfd6fa6e812e8bdd4b4f17349196632bff20736035475898eed432f5abec: Status 404 returned error can't find the container with id a4e0bfd6fa6e812e8bdd4b4f17349196632bff20736035475898eed432f5abec
Jul 21 12:12:25.750526 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.750438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7" event={"ID":"f6b7174f-ac0b-4698-b020-fdb8ea0913cb","Type":"ContainerStarted","Data":"fb3ad231ab7802ce7a08080b9e34a2c5f855d90bcc14f0b892369aa9b34c673f"}
Jul 21 12:12:25.750526 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.750497    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7" event={"ID":"f6b7174f-ac0b-4698-b020-fdb8ea0913cb","Type":"ContainerStarted","Data":"e9a37f18e2cf53d626a381a707432632fd0f764e286715758d8240307e5e0dc1"}
Jul 21 12:12:25.750526 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.750515    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7" event={"ID":"f6b7174f-ac0b-4698-b020-fdb8ea0913cb","Type":"ContainerStarted","Data":"63ed46a1904c8b7095cb14de9f69342354833d0f9dbd424989a4756ea3f8c4f0"}
Jul 21 12:12:25.752474 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.752441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lpnp2" event={"ID":"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d","Type":"ContainerStarted","Data":"65635c127bbc9b77646618d1f1cf173330632175988293e571c3f764e94fbaf1"}
Jul 21 12:12:25.754962 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.754904    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4tb74" event={"ID":"fa35c26d-86c4-456a-862f-fdc14069ec25","Type":"ContainerStarted","Data":"840c081b28bafea9e7451fd50d4b3a3fd435eb4b21ee9565de8c77c174ab2a67"}
Jul 21 12:12:25.757730 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.757703    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdb7w" event={"ID":"7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee","Type":"ContainerStarted","Data":"a766574dbd440de58a35f4ce0f430c6881d0e558f6330e08cb4753ee33c55857"}
Jul 21 12:12:25.759579 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.759550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q7tnn" event={"ID":"5b6c7079-8edc-469b-af1c-61f5ccfa0511","Type":"ContainerStarted","Data":"a4e0bfd6fa6e812e8bdd4b4f17349196632bff20736035475898eed432f5abec"}
Jul 21 12:12:25.769968 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.769918    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dcdb5fbf6-k8rh7" podStartSLOduration=3.014599278 podStartE2EDuration="8.76990041s" totalImagesPullingTime="5.755301132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:25.769707564 +0000 UTC m=+69.828138568" watchObservedRunningTime="2026-07-21 12:12:25.76990041 +0000 UTC m=+69.828331416"
Jul 21 12:12:25.789889 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:25.789185    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mdb7w" podStartSLOduration=17.732144555 podStartE2EDuration="31.78916493s" totalImagesPullingTime="14.057020375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:25.788234886 +0000 UTC m=+69.846665892" watchObservedRunningTime="2026-07-21 12:12:25.78916493 +0000 UTC m=+69.847595935"
Jul 21 12:12:26.766136 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:26.766051    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4tb74" event={"ID":"fa35c26d-86c4-456a-862f-fdc14069ec25","Type":"ContainerStarted","Data":"427eef8c5afb9e8520f3fa1731099b0b067f9c77b686399178be25c266792bd6"}
Jul 21 12:12:26.766528 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:26.766157    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4tb74" event={"ID":"fa35c26d-86c4-456a-862f-fdc14069ec25","Type":"ContainerStarted","Data":"c0e2fa9b08fe99ac0499cb7cbba5e0f4be21886c8648112a88222a8ccb397e1f"}
Jul 21 12:12:26.784147 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:26.784089    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4tb74" podStartSLOduration=69.707808763 podStartE2EDuration="1m10.78407104s" totalImagesPullingTime="1.076262277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:26.783166396 +0000 UTC m=+70.841597394" watchObservedRunningTime="2026-07-21 12:12:26.78407104 +0000 UTC m=+70.842502043"
Jul 21 12:12:30.781741 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:30.781706    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q7tnn" event={"ID":"5b6c7079-8edc-469b-af1c-61f5ccfa0511","Type":"ContainerStarted","Data":"b1c1065eadba3ebed32a633668c1a4e05d31ffca42479a6e599e7b0b3980db70"}
Jul 21 12:12:30.783239 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:30.783209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lpnp2" event={"ID":"b788c24c-d635-4fcb-9f86-4ae4b2a1f56d","Type":"ContainerStarted","Data":"8ba5f32d5496d7f48aa268b3929e08179fb6b4bed77d6812bff6b255c1ca35d1"}
Jul 21 12:12:30.783388 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:30.783272    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:12:30.802035 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:30.799324    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q7tnn" podStartSLOduration=70.342403228 podStartE2EDuration="1m14.799304888s" totalImagesPullingTime="4.45690166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:30.797553081 +0000 UTC m=+74.855984087" watchObservedRunningTime="2026-07-21 12:12:30.799304888 +0000 UTC m=+74.857735893"
Jul 21 12:12:30.815435 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:30.815392    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lpnp2" podStartSLOduration=70.329634922 podStartE2EDuration="1m14.815377284s" totalImagesPullingTime="4.485742362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:30.814731098 +0000 UTC m=+74.873162102" watchObservedRunningTime="2026-07-21 12:12:30.815377284 +0000 UTC m=+74.873808287"
Jul 21 12:12:37.982617 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.982500    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fd689dfd5-f6gv7"]
Jul 21 12:12:37.985531 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.985513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:37.988247 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.988224    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 12:12:37.988247 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.988238    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 12:12:37.988428 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.988223    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 12:12:37.989466 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.989444    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 12:12:37.989584 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.989468    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 12:12:37.989584 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.989492    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kkd5b\""
Jul 21 12:12:37.989584 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.989544    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 12:12:37.989757 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.989747    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 12:12:37.993656 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.993629    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 21 12:12:37.997604 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:37.997582    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fd689dfd5-f6gv7"]
Jul 21 12:12:38.128008 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.127972    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-oauth-serving-cert\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.128008 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.128011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc7bz\" (UniqueName: \"kubernetes.io/projected/f56a8e6d-f711-44e2-9f8c-288576c08088-kube-api-access-zc7bz\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.128263 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.128053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-serving-cert\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.128263 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.128098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-service-ca\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.128263 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.128133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-console-config\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.128263 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.128154    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-oauth-config\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.128263 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.128171    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-trusted-ca-bundle\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.150798 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.150745    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-66whn"]
Jul 21 12:12:38.154496 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.154474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.157259 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.157237    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 12:12:38.157403 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.157379    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 12:12:38.157519 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.157424    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 12:12:38.157519 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.157252    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 12:12:38.157627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.157312    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 12:12:38.157627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.157242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cj58p\""
Jul 21 12:12:38.157721 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.157664    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 12:12:38.229270 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-accelerators-collector-config\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.229476 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc7bz\" (UniqueName: \"kubernetes.io/projected/f56a8e6d-f711-44e2-9f8c-288576c08088-kube-api-access-zc7bz\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.229476 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229346    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-tls\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.229476 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-textfile\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.229634 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9f9b59c-5d84-4796-9691-4248b1a94346-metrics-client-ca\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.229634 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-serving-cert\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.229634 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.229804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-service-ca\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.229804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229668    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wflg9\" (UniqueName: \"kubernetes.io/projected/b9f9b59c-5d84-4796-9691-4248b1a94346-kube-api-access-wflg9\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.229804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229698    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-console-config\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.229804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229727    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-oauth-config\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.229804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229755    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-trusted-ca-bundle\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.230053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229803    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-wtmp\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.230053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229833    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b9f9b59c-5d84-4796-9691-4248b1a94346-root\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.230053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9f9b59c-5d84-4796-9691-4248b1a94346-sys\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.230053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.229901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-oauth-serving-cert\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.230528 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.230466    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-service-ca\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.230663 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.230619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-oauth-serving-cert\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.230663 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.230657    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-console-config\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.230989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.230962    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-trusted-ca-bundle\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.233370 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.233317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-serving-cert\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.236215 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.236190    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-oauth-config\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.237422 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.237400    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc7bz\" (UniqueName: \"kubernetes.io/projected/f56a8e6d-f711-44e2-9f8c-288576c08088-kube-api-access-zc7bz\") pod \"console-7fd689dfd5-f6gv7\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") " pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.295405 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.295362    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:38.330935 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.330896    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-accelerators-collector-config\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.330935 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.330945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-tls\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.331181 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.330973    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-textfile\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.331181 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.331009    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9f9b59c-5d84-4796-9691-4248b1a94346-metrics-client-ca\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.331181 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.331042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.331181 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.331079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wflg9\" (UniqueName: \"kubernetes.io/projected/b9f9b59c-5d84-4796-9691-4248b1a94346-kube-api-access-wflg9\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.331181 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.331120    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-wtmp\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.331181 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.331148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b9f9b59c-5d84-4796-9691-4248b1a94346-root\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.331181 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.331172    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9f9b59c-5d84-4796-9691-4248b1a94346-sys\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.332138 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.331266    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9f9b59c-5d84-4796-9691-4248b1a94346-sys\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.332138 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:12:38.331720    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 21 12:12:38.332138 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:12:38.331823    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-tls podName:b9f9b59c-5d84-4796-9691-4248b1a94346 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:38.831801498 +0000 UTC m=+82.890232495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-tls") pod "node-exporter-66whn" (UID: "b9f9b59c-5d84-4796-9691-4248b1a94346") : secret "node-exporter-tls" not found
Jul 21 12:12:38.332138 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.331964    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9f9b59c-5d84-4796-9691-4248b1a94346-metrics-client-ca\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.332347 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.332187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-wtmp\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.332347 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.332245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-textfile\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.332347 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.332268    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-accelerators-collector-config\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.332347 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.332316    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b9f9b59c-5d84-4796-9691-4248b1a94346-root\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.335097 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.335042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.342977 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.342949    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wflg9\" (UniqueName: \"kubernetes.io/projected/b9f9b59c-5d84-4796-9691-4248b1a94346-kube-api-access-wflg9\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.457693 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.457657    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fd689dfd5-f6gv7"]
Jul 21 12:12:38.461025 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:38.460994    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf56a8e6d_f711_44e2_9f8c_288576c08088.slice/crio-e9cdd0c01e8cdfaa932171b0010b392ec155a92fecab6c58b1464a7ce7374040 WatchSource:0}: Error finding container e9cdd0c01e8cdfaa932171b0010b392ec155a92fecab6c58b1464a7ce7374040: Status 404 returned error can't find the container with id e9cdd0c01e8cdfaa932171b0010b392ec155a92fecab6c58b1464a7ce7374040
Jul 21 12:12:38.808739 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.808695    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd689dfd5-f6gv7" event={"ID":"f56a8e6d-f711-44e2-9f8c-288576c08088","Type":"ContainerStarted","Data":"e9cdd0c01e8cdfaa932171b0010b392ec155a92fecab6c58b1464a7ce7374040"}
Jul 21 12:12:38.835627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.835587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-tls\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:38.838052 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:38.838027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b9f9b59c-5d84-4796-9691-4248b1a94346-node-exporter-tls\") pod \"node-exporter-66whn\" (UID: \"b9f9b59c-5d84-4796-9691-4248b1a94346\") " pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:39.066250 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.066216    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-66whn"
Jul 21 12:12:39.082632 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:39.082600    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f9b59c_5d84_4796_9691_4248b1a94346.slice/crio-b59866c2cbdc11d77ce8e834ef1b69afe8ad93d2621899264ebcc5dae5a840b2 WatchSource:0}: Error finding container b59866c2cbdc11d77ce8e834ef1b69afe8ad93d2621899264ebcc5dae5a840b2: Status 404 returned error can't find the container with id b59866c2cbdc11d77ce8e834ef1b69afe8ad93d2621899264ebcc5dae5a840b2
Jul 21 12:12:39.172005 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.171967    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:12:39.177200 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.177174    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.179788 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.179745    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mx59h\""
Jul 21 12:12:39.179788 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.179762    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 21 12:12:39.180008 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.179754    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 21 12:12:39.180123 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.180016    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 21 12:12:39.180248 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.180233    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 21 12:12:39.180309 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.180291    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 21 12:12:39.180951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.180805    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 21 12:12:39.181059 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.181011    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 21 12:12:39.181059 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.181024    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 21 12:12:39.181059 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.181049    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 21 12:12:39.191782 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.191754    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:12:39.239269 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239238    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239441 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239441 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239441 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239343    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239441 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239373    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vghx\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-kube-api-access-7vghx\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239441 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-web-config\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239448    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-out\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239477    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239506    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-volume\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239530    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.239686 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.239616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.340173 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.340083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-out\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.340173 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.340134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.340173 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.340160    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-volume\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.340446 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.340182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.340446 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.340213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.340710    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.340761    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.341684    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.341838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.341867    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.341892    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.341932    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.341969    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vghx\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-kube-api-access-7vghx\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.342004    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-web-config\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:12:39.342445    2562 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:12:39.342527    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls podName:d364de4d-964e-4b2d-bd91-8ca08c445fd8 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:39.842505616 +0000 UTC m=+83.900936612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8") : secret "alertmanager-main-tls" not found
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.342707    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:12:39.342873    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-trusted-ca-bundle podName:d364de4d-964e-4b2d-bd91-8ca08c445fd8 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:39.842857241 +0000 UTC m=+83.901288237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8") : configmap references non-existent config key: ca-bundle.crt
Jul 21 12:12:39.343560 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.343498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.344653 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.343803    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-out\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.344653 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.344114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.345322 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.345277    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.345447 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.345424    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-volume\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.345514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.345443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.346325 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.346300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.347311 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.347272    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-web-config\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.350961 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.350937    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vghx\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-kube-api-access-7vghx\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.814565 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.814471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66whn" event={"ID":"b9f9b59c-5d84-4796-9691-4248b1a94346","Type":"ContainerStarted","Data":"b59866c2cbdc11d77ce8e834ef1b69afe8ad93d2621899264ebcc5dae5a840b2"}
Jul 21 12:12:39.847157 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.847117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.847330 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.847169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:39.847330 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:12:39.847306    2562 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 21 12:12:39.847435 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:12:39.847363    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls podName:d364de4d-964e-4b2d-bd91-8ca08c445fd8 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:40.847349896 +0000 UTC m=+84.905780878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8") : secret "alertmanager-main-tls" not found
Jul 21 12:12:39.848493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:39.848466    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:40.819684 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:40.819609    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66whn" event={"ID":"b9f9b59c-5d84-4796-9691-4248b1a94346","Type":"ContainerStarted","Data":"c8e122ebcf40d740c813c22a8db1e6e0a3272bbc88f1f81420bfaa674d813368"}
Jul 21 12:12:40.857529 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:40.857493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:40.860238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:40.860208    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:40.989125 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:40.989090    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:12:41.407444 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:41.407418    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:12:41.408868 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:41.408843    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd364de4d_964e_4b2d_bd91_8ca08c445fd8.slice/crio-b9e07b8f01056d1d957c3778ff213e11552b532954b851c9a783d979e44f52d1 WatchSource:0}: Error finding container b9e07b8f01056d1d957c3778ff213e11552b532954b851c9a783d979e44f52d1: Status 404 returned error can't find the container with id b9e07b8f01056d1d957c3778ff213e11552b532954b851c9a783d979e44f52d1
Jul 21 12:12:41.823839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:41.823803    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerStarted","Data":"b9e07b8f01056d1d957c3778ff213e11552b532954b851c9a783d979e44f52d1"}
Jul 21 12:12:41.825299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:41.825273    2562 generic.go:356] "Generic (PLEG): container finished" podID="b9f9b59c-5d84-4796-9691-4248b1a94346" containerID="c8e122ebcf40d740c813c22a8db1e6e0a3272bbc88f1f81420bfaa674d813368" exitCode=0
Jul 21 12:12:41.825436 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:41.825381    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66whn" event={"ID":"b9f9b59c-5d84-4796-9691-4248b1a94346","Type":"ContainerDied","Data":"c8e122ebcf40d740c813c22a8db1e6e0a3272bbc88f1f81420bfaa674d813368"}
Jul 21 12:12:41.826852 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:41.826830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd689dfd5-f6gv7" event={"ID":"f56a8e6d-f711-44e2-9f8c-288576c08088","Type":"ContainerStarted","Data":"b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815"}
Jul 21 12:12:41.863267 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:41.863223    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fd689dfd5-f6gv7" podStartSLOduration=2.004697929 podStartE2EDuration="4.863206812s" totalImagesPullingTime="2.858508883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:41.862459658 +0000 UTC m=+85.920890664" watchObservedRunningTime="2026-07-21 12:12:41.863206812 +0000 UTC m=+85.921637815"
Jul 21 12:12:42.832725 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:42.832619    2562 generic.go:356] "Generic (PLEG): container finished" podID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerID="faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a" exitCode=0
Jul 21 12:12:42.832725 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:42.832710    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerDied","Data":"faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a"}
Jul 21 12:12:42.836693 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:42.836651    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66whn" event={"ID":"b9f9b59c-5d84-4796-9691-4248b1a94346","Type":"ContainerStarted","Data":"860fe935888779476b18f5d51c28fcb42a871e881ebcba80e24fa5054ca65c62"}
Jul 21 12:12:42.836833 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:42.836721    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66whn" event={"ID":"b9f9b59c-5d84-4796-9691-4248b1a94346","Type":"ContainerStarted","Data":"9958bc988d00c1dea04670e3a4558ed1f1a352088de5cfafb89f9dbd48df572b"}
Jul 21 12:12:42.888630 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:42.888574    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-66whn" podStartSLOduration=3.843408063 podStartE2EDuration="4.888552232s" totalImagesPullingTime="968.075754ms" totalInitContainerRuntime="77.068415ms" isStatefulPod=true podCreationTimestamp="2026-07-21 12:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:42.886904536 +0000 UTC m=+86.945335566" watchObservedRunningTime="2026-07-21 12:12:42.888552232 +0000 UTC m=+86.946983238"
Jul 21 12:12:44.847487 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:44.847407    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerStarted","Data":"b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f"}
Jul 21 12:12:44.847826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:44.847503    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerStarted","Data":"a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a"}
Jul 21 12:12:44.847826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:44.847531    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerStarted","Data":"4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f"}
Jul 21 12:12:44.847826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:44.847554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerStarted","Data":"83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a"}
Jul 21 12:12:44.847826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:44.847577    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerStarted","Data":"e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d"}
Jul 21 12:12:45.598283 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:45.598254    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fd689dfd5-f6gv7"]
Jul 21 12:12:46.857806 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:46.857754    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerStarted","Data":"b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7"}
Jul 21 12:12:46.888879 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:46.888812    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.906477907 podStartE2EDuration="7.888794142s" totalImagesPullingTime="3.961740209s" totalInitContainerRuntime="20.576026ms" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:46.88715206 +0000 UTC m=+90.945583064" watchObservedRunningTime="2026-07-21 12:12:46.888794142 +0000 UTC m=+90.947225143"
Jul 21 12:12:48.296387 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:48.296347    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:12:50.751834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:50.751749    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74fd9659cc-m76jm"]
Jul 21 12:12:50.890065 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:50.890031    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74fd9659cc-m76jm"]
Jul 21 12:12:50.890221 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:50.890145    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.041197 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.041113    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-config\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.041197 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.041153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-oauth-serving-cert\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.041197 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.041175    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jh2n\" (UniqueName: \"kubernetes.io/projected/41c2be73-10c1-434e-8e51-fd74f138fd0f-kube-api-access-7jh2n\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.041406 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.041260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-serving-cert\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.041406 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.041309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-service-ca\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.041470 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.041408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-oauth-config\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.041470 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.041432    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-trusted-ca-bundle\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.142493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.142448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-oauth-config\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.142493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.142493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-trusted-ca-bundle\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.142726 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.142529    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-config\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.142726 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.142552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-oauth-serving-cert\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.142726 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.142577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jh2n\" (UniqueName: \"kubernetes.io/projected/41c2be73-10c1-434e-8e51-fd74f138fd0f-kube-api-access-7jh2n\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.142726 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.142615    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-serving-cert\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.142726 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.142634    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-service-ca\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.143523 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.143494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-config\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.143649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.143525    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-trusted-ca-bundle\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.143734 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.143711    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-service-ca\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.143796 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.143742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-oauth-serving-cert\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.145248 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.145218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-oauth-config\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.145350 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.145329    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-serving-cert\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.150562 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.150541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jh2n\" (UniqueName: \"kubernetes.io/projected/41c2be73-10c1-434e-8e51-fd74f138fd0f-kube-api-access-7jh2n\") pod \"console-74fd9659cc-m76jm\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") " pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.198757 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.198713    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:12:51.354115 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.354084    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74fd9659cc-m76jm"]
Jul 21 12:12:51.357410 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:12:51.357382    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c2be73_10c1_434e_8e51_fd74f138fd0f.slice/crio-4a1802c6cf5322f3110899990918a213a1c3bad1a640e94018d91825879ceda8 WatchSource:0}: Error finding container 4a1802c6cf5322f3110899990918a213a1c3bad1a640e94018d91825879ceda8: Status 404 returned error can't find the container with id 4a1802c6cf5322f3110899990918a213a1c3bad1a640e94018d91825879ceda8
Jul 21 12:12:51.878378 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.878344    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74fd9659cc-m76jm" event={"ID":"41c2be73-10c1-434e-8e51-fd74f138fd0f","Type":"ContainerStarted","Data":"a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72"}
Jul 21 12:12:51.878793 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.878403    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74fd9659cc-m76jm" event={"ID":"41c2be73-10c1-434e-8e51-fd74f138fd0f","Type":"ContainerStarted","Data":"4a1802c6cf5322f3110899990918a213a1c3bad1a640e94018d91825879ceda8"}
Jul 21 12:12:51.903205 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:12:51.903159    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74fd9659cc-m76jm" podStartSLOduration=1.903143578 podStartE2EDuration="1.903143578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:51.902366458 +0000 UTC m=+95.960797461" watchObservedRunningTime="2026-07-21 12:12:51.903143578 +0000 UTC m=+95.961574583"
Jul 21 12:13:01.199092 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:01.199051    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:13:01.199516 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:01.199119    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:13:01.203861 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:01.203839    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:13:01.788926 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:01.788895    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lpnp2"
Jul 21 12:13:01.916334 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:01.916301    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:13:10.619769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.619720    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7fd689dfd5-f6gv7" podUID="f56a8e6d-f711-44e2-9f8c-288576c08088" containerName="console" containerID="cri-o://b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815" gracePeriod=15
Jul 21 12:13:10.629041 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.629015    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tb2sh_ce7c7b6d-ce2c-4026-83de-c59e7d3abae9/serve-healthcheck-canary/0.log"
Jul 21 12:13:10.863738 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.863713    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:13:10.882197 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882128    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc7bz\" (UniqueName: \"kubernetes.io/projected/f56a8e6d-f711-44e2-9f8c-288576c08088-kube-api-access-zc7bz\") pod \"f56a8e6d-f711-44e2-9f8c-288576c08088\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") "
Jul 21 12:13:10.882197 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882163    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-serving-cert\") pod \"f56a8e6d-f711-44e2-9f8c-288576c08088\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") "
Jul 21 12:13:10.882387 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882200    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-oauth-config\") pod \"f56a8e6d-f711-44e2-9f8c-288576c08088\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") "
Jul 21 12:13:10.882451 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882384    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-service-ca\") pod \"f56a8e6d-f711-44e2-9f8c-288576c08088\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") "
Jul 21 12:13:10.882451 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882425    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-trusted-ca-bundle\") pod \"f56a8e6d-f711-44e2-9f8c-288576c08088\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") "
Jul 21 12:13:10.882547 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882458    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-console-config\") pod \"f56a8e6d-f711-44e2-9f8c-288576c08088\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") "
Jul 21 12:13:10.882547 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882528    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-oauth-serving-cert\") pod \"f56a8e6d-f711-44e2-9f8c-288576c08088\" (UID: \"f56a8e6d-f711-44e2-9f8c-288576c08088\") "
Jul 21 12:13:10.883232 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882733    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-service-ca" (OuterVolumeSpecName: "service-ca") pod "f56a8e6d-f711-44e2-9f8c-288576c08088" (UID: "f56a8e6d-f711-44e2-9f8c-288576c08088"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:13:10.883232 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882862    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f56a8e6d-f711-44e2-9f8c-288576c08088" (UID: "f56a8e6d-f711-44e2-9f8c-288576c08088"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:13:10.883232 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.882934    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-console-config" (OuterVolumeSpecName: "console-config") pod "f56a8e6d-f711-44e2-9f8c-288576c08088" (UID: "f56a8e6d-f711-44e2-9f8c-288576c08088"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:13:10.883232 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.883190    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f56a8e6d-f711-44e2-9f8c-288576c08088" (UID: "f56a8e6d-f711-44e2-9f8c-288576c08088"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:13:10.885040 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.884981    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f56a8e6d-f711-44e2-9f8c-288576c08088" (UID: "f56a8e6d-f711-44e2-9f8c-288576c08088"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:13:10.885210 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.885114    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f56a8e6d-f711-44e2-9f8c-288576c08088" (UID: "f56a8e6d-f711-44e2-9f8c-288576c08088"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:13:10.885210 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.885185    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f56a8e6d-f711-44e2-9f8c-288576c08088-kube-api-access-zc7bz" (OuterVolumeSpecName: "kube-api-access-zc7bz") pod "f56a8e6d-f711-44e2-9f8c-288576c08088" (UID: "f56a8e6d-f711-44e2-9f8c-288576c08088"). InnerVolumeSpecName "kube-api-access-zc7bz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:13:10.944565 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.944532    2562 generic.go:356] "Generic (PLEG): container finished" podID="f56a8e6d-f711-44e2-9f8c-288576c08088" containerID="b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815" exitCode=0
Jul 21 12:13:10.944698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.944576    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fd689dfd5-f6gv7"
Jul 21 12:13:10.944698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.944618    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd689dfd5-f6gv7" event={"ID":"f56a8e6d-f711-44e2-9f8c-288576c08088","Type":"ContainerDied","Data":"b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815"}
Jul 21 12:13:10.944698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.944669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fd689dfd5-f6gv7" event={"ID":"f56a8e6d-f711-44e2-9f8c-288576c08088","Type":"ContainerDied","Data":"e9cdd0c01e8cdfaa932171b0010b392ec155a92fecab6c58b1464a7ce7374040"}
Jul 21 12:13:10.944698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.944695    2562 scope.go:117] "RemoveContainer" containerID="b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815"
Jul 21 12:13:10.953170 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.953154    2562 scope.go:117] "RemoveContainer" containerID="b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815"
Jul 21 12:13:10.953411 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:13:10.953393    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815\": container with ID starting with b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815 not found: ID does not exist" containerID="b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815"
Jul 21 12:13:10.953455 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.953425    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815"} err="failed to get container status \"b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815\": rpc error: code = NotFound desc = could not find container \"b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815\": container with ID starting with b422648ce34673e1cf114d23bbd5a986d3546739e26992dbfa0838206d7f5815 not found: ID does not exist"
Jul 21 12:13:10.963985 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.963960    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fd689dfd5-f6gv7"]
Jul 21 12:13:10.969672 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.969653    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fd689dfd5-f6gv7"]
Jul 21 12:13:10.983880 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.983861    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:10.983880 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.983879    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f56a8e6d-f711-44e2-9f8c-288576c08088-console-oauth-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:10.983978 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.983889    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:10.983978 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.983899    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-trusted-ca-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:10.983978 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.983907    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-console-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:10.983978 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.983915    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f56a8e6d-f711-44e2-9f8c-288576c08088-oauth-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:10.983978 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:10.983925    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc7bz\" (UniqueName: \"kubernetes.io/projected/f56a8e6d-f711-44e2-9f8c-288576c08088-kube-api-access-zc7bz\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:12.416187 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:12.416152    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f56a8e6d-f711-44e2-9f8c-288576c08088" path="/var/lib/kubelet/pods/f56a8e6d-f711-44e2-9f8c-288576c08088/volumes"
Jul 21 12:13:58.491960 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:58.491922    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:13:58.492413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:58.492241    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="alertmanager" containerID="cri-o://e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d" gracePeriod=120
Jul 21 12:13:58.492413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:58.492276    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy-metric" containerID="cri-o://b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f" gracePeriod=120
Jul 21 12:13:58.492413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:58.492337    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="prom-label-proxy" containerID="cri-o://b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7" gracePeriod=120
Jul 21 12:13:58.492413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:58.492351    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy-web" containerID="cri-o://4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f" gracePeriod=120
Jul 21 12:13:58.492645 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:58.492362    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy" containerID="cri-o://a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a" gracePeriod=120
Jul 21 12:13:58.492645 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:58.492555    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="config-reloader" containerID="cri-o://83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a" gracePeriod=120
Jul 21 12:13:59.111990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.111959    2562 generic.go:356] "Generic (PLEG): container finished" podID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerID="b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7" exitCode=0
Jul 21 12:13:59.111990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.111985    2562 generic.go:356] "Generic (PLEG): container finished" podID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerID="b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f" exitCode=0
Jul 21 12:13:59.111990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.111993    2562 generic.go:356] "Generic (PLEG): container finished" podID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerID="a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a" exitCode=0
Jul 21 12:13:59.111990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.112001    2562 generic.go:356] "Generic (PLEG): container finished" podID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerID="83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a" exitCode=0
Jul 21 12:13:59.111990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.112010    2562 generic.go:356] "Generic (PLEG): container finished" podID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerID="e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d" exitCode=0
Jul 21 12:13:59.112299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.112035    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerDied","Data":"b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7"}
Jul 21 12:13:59.112299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.112106    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerDied","Data":"b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f"}
Jul 21 12:13:59.112299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.112133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerDied","Data":"a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a"}
Jul 21 12:13:59.112299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.112158    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerDied","Data":"83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a"}
Jul 21 12:13:59.112299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.112184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerDied","Data":"e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d"}
Jul 21 12:13:59.723616 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.723592    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:13:59.851690 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.851644    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.851867 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.851718    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-main-db\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.851867 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.851743    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-out\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.851867 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.851821    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-web-config\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.851867 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.851850    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-volume\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.852092 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.851875    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-web\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.852092 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.851903    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vghx\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-kube-api-access-7vghx\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.852092 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.851937    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.852092 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.851973    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-tls-assets\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.852092 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.852005    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.852092 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.852038    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-trusted-ca-bundle\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.852092 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.852064    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-metrics-client-ca\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.852092 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.852089    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-cluster-tls-config\") pod \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\" (UID: \"d364de4d-964e-4b2d-bd91-8ca08c445fd8\") "
Jul 21 12:13:59.852524 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.852098    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:13:59.852524 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.852301    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-main-db\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.852832 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.852797    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:13:59.853457 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.853428    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:13:59.855517 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.855481    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:13:59.855879 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.855834    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-volume" (OuterVolumeSpecName: "config-volume") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:13:59.855879 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.855853    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-out" (OuterVolumeSpecName: "config-out") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:13:59.856028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.855876    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:13:59.856288 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.856265    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:13:59.856363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.856286    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:13:59.856363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.856324    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:13:59.856828 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.856807    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-kube-api-access-7vghx" (OuterVolumeSpecName: "kube-api-access-7vghx") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "kube-api-access-7vghx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:13:59.859436 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.859378    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:13:59.865087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.865063    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-web-config" (OuterVolumeSpecName: "web-config") pod "d364de4d-964e-4b2d-bd91-8ca08c445fd8" (UID: "d364de4d-964e-4b2d-bd91-8ca08c445fd8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:13:59.953659 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953621    2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-cluster-tls-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953659 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953648    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-main-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953659 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953658    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-out\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953686    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-web-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953695    2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-config-volume\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953705    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953714    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vghx\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-kube-api-access-7vghx\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953723    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953731    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d364de4d-964e-4b2d-bd91-8ca08c445fd8-tls-assets\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953740    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d364de4d-964e-4b2d-bd91-8ca08c445fd8-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953749    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:13:59.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:13:59.953757    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d364de4d-964e-4b2d-bd91-8ca08c445fd8-metrics-client-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:14:00.118861 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.118766    2562 generic.go:356] "Generic (PLEG): container finished" podID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerID="4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f" exitCode=0
Jul 21 12:14:00.118861 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.118811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerDied","Data":"4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f"}
Jul 21 12:14:00.119021 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.118878    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.119021 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.118901    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d364de4d-964e-4b2d-bd91-8ca08c445fd8","Type":"ContainerDied","Data":"b9e07b8f01056d1d957c3778ff213e11552b532954b851c9a783d979e44f52d1"}
Jul 21 12:14:00.119021 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.118951    2562 scope.go:117] "RemoveContainer" containerID="b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7"
Jul 21 12:14:00.127392 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.127376    2562 scope.go:117] "RemoveContainer" containerID="b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f"
Jul 21 12:14:00.134249 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.134233    2562 scope.go:117] "RemoveContainer" containerID="a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a"
Jul 21 12:14:00.140665 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.140648    2562 scope.go:117] "RemoveContainer" containerID="4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f"
Jul 21 12:14:00.142547 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.142525    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:14:00.147843 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.147827    2562 scope.go:117] "RemoveContainer" containerID="83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a"
Jul 21 12:14:00.148234 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.148216    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:14:00.154385 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.154369    2562 scope.go:117] "RemoveContainer" containerID="e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d"
Jul 21 12:14:00.160971 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.160950    2562 scope.go:117] "RemoveContainer" containerID="faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a"
Jul 21 12:14:00.167467 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.167454    2562 scope.go:117] "RemoveContainer" containerID="b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7"
Jul 21 12:14:00.167705 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:14:00.167686    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7\": container with ID starting with b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7 not found: ID does not exist" containerID="b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7"
Jul 21 12:14:00.167745 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.167719    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7"} err="failed to get container status \"b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7\": rpc error: code = NotFound desc = could not find container \"b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7\": container with ID starting with b85131eb61818233202c2581993a5f1d8ec966c03bfcd5c7080ce395f7dddae7 not found: ID does not exist"
Jul 21 12:14:00.167745 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.167738    2562 scope.go:117] "RemoveContainer" containerID="b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f"
Jul 21 12:14:00.167999 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:14:00.167980    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f\": container with ID starting with b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f not found: ID does not exist" containerID="b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f"
Jul 21 12:14:00.168038 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.168007    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f"} err="failed to get container status \"b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f\": rpc error: code = NotFound desc = could not find container \"b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f\": container with ID starting with b895ca23d6094ea035b3ce54d7537884db3a2262e1e61f64ae7c886a6b3b3f3f not found: ID does not exist"
Jul 21 12:14:00.168038 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.168022    2562 scope.go:117] "RemoveContainer" containerID="a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a"
Jul 21 12:14:00.168277 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:14:00.168261    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a\": container with ID starting with a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a not found: ID does not exist" containerID="a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a"
Jul 21 12:14:00.168320 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.168283    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a"} err="failed to get container status \"a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a\": rpc error: code = NotFound desc = could not find container \"a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a\": container with ID starting with a52484dcb2154e58bd08b61743328f63844d471815433197e9bc2322207a5a8a not found: ID does not exist"
Jul 21 12:14:00.168320 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.168295    2562 scope.go:117] "RemoveContainer" containerID="4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f"
Jul 21 12:14:00.168534 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:14:00.168520    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f\": container with ID starting with 4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f not found: ID does not exist" containerID="4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f"
Jul 21 12:14:00.168579 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.168536    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f"} err="failed to get container status \"4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f\": rpc error: code = NotFound desc = could not find container \"4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f\": container with ID starting with 4349761b412c5988e4bcdaf682e7b4157a29ad2bef7168896653b52443c2c51f not found: ID does not exist"
Jul 21 12:14:00.168579 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.168550    2562 scope.go:117] "RemoveContainer" containerID="83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a"
Jul 21 12:14:00.168767 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:14:00.168750    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a\": container with ID starting with 83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a not found: ID does not exist" containerID="83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a"
Jul 21 12:14:00.168832 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.168787    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a"} err="failed to get container status \"83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a\": rpc error: code = NotFound desc = could not find container \"83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a\": container with ID starting with 83975004ff3ff727717d6a2cf18419924508f56e5bfa45ad103d60cfd363cc9a not found: ID does not exist"
Jul 21 12:14:00.168832 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.168802    2562 scope.go:117] "RemoveContainer" containerID="e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d"
Jul 21 12:14:00.169033 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:14:00.169017    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d\": container with ID starting with e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d not found: ID does not exist" containerID="e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d"
Jul 21 12:14:00.169082 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.169036    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d"} err="failed to get container status \"e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d\": rpc error: code = NotFound desc = could not find container \"e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d\": container with ID starting with e95d3a4fb7ccb204504cf7fe87d9f2fdc65a86e11bd01b0a315e17990b8e413d not found: ID does not exist"
Jul 21 12:14:00.169082 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.169049    2562 scope.go:117] "RemoveContainer" containerID="faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a"
Jul 21 12:14:00.169273 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:14:00.169252    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a\": container with ID starting with faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a not found: ID does not exist" containerID="faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a"
Jul 21 12:14:00.169318 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.169280    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a"} err="failed to get container status \"faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a\": rpc error: code = NotFound desc = could not find container \"faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a\": container with ID starting with faee95f030526c5ffe989ca98698d1ddc0a3dca07ca5d1066cf5b622331e4f5a not found: ID does not exist"
Jul 21 12:14:00.174348 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174329    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:14:00.174546 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174535    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56a8e6d-f711-44e2-9f8c-288576c08088" containerName="console"
Jul 21 12:14:00.174592 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174548    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56a8e6d-f711-44e2-9f8c-288576c08088" containerName="console"
Jul 21 12:14:00.174592 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174556    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy"
Jul 21 12:14:00.174592 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174562    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy"
Jul 21 12:14:00.174592 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174570    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="alertmanager"
Jul 21 12:14:00.174592 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174574    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="alertmanager"
Jul 21 12:14:00.174592 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174581    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="prom-label-proxy"
Jul 21 12:14:00.174592 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174587    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="prom-label-proxy"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174598    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy-metric"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174603    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy-metric"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174610    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="init-config-reloader"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174615    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="init-config-reloader"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174621    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="config-reloader"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174626    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="config-reloader"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174633    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy-web"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174638    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy-web"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174675    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="config-reloader"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174683    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="alertmanager"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174688    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy-web"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174693    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174699    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f56a8e6d-f711-44e2-9f8c-288576c08088" containerName="console"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174704    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="kube-rbac-proxy-metric"
Jul 21 12:14:00.174881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.174710    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" containerName="prom-label-proxy"
Jul 21 12:14:00.178848 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.178833    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.181346 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.181325    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 21 12:14:00.181499 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.181349    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 21 12:14:00.181499 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.181474    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 21 12:14:00.181614 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.181566    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 21 12:14:00.181666 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.181612    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 21 12:14:00.181666 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.181632    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 21 12:14:00.181881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.181864    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 21 12:14:00.181943 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.181864    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 21 12:14:00.181996 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.181886    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mx59h\""
Jul 21 12:14:00.187304 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.187284    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 21 12:14:00.192320 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.192303    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:14:00.356222 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356175    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356222 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356221    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356460 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356242    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356460 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-web-config\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356460 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356307    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356460 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356460 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356460 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356660 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356660 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64q2v\" (UniqueName: \"kubernetes.io/projected/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-kube-api-access-64q2v\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356660 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356660 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.356660 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.356577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-config-out\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.415625 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.415540    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d364de4d-964e-4b2d-bd91-8ca08c445fd8" path="/var/lib/kubelet/pods/d364de4d-964e-4b2d-bd91-8ca08c445fd8/volumes"
Jul 21 12:14:00.457156 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457127    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457319 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-config-out\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457319 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457189    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457319 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457206    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457319 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457319 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457249    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-web-config\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457319 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457319 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457319 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457319    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457715 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457346    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457715 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457369    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457715 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64q2v\" (UniqueName: \"kubernetes.io/projected/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-kube-api-access-64q2v\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457715 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.457715 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.457665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.458343 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.458308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.460166 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.460137    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-config-out\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.460339 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.460307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-web-config\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.460509 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.460485    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.460671 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.460621    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.460762 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.460734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.460889 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.460867    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.461008 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.460986    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.461195 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.461176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.461253 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.461177    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.462306 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.462286    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.468422 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.468395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64q2v\" (UniqueName: \"kubernetes.io/projected/1f1f12cb-ae83-409a-90ea-fa45b7f7434f-kube-api-access-64q2v\") pod \"alertmanager-main-0\" (UID: \"1f1f12cb-ae83-409a-90ea-fa45b7f7434f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.488874 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.488851    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:14:00.647460 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:00.647434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:14:00.649854 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:14:00.649824    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f1f12cb_ae83_409a_90ea_fa45b7f7434f.slice/crio-39e20f2343e756fc30c65087356964515a89bd86429bfbf7262d6cda5a8f5865 WatchSource:0}: Error finding container 39e20f2343e756fc30c65087356964515a89bd86429bfbf7262d6cda5a8f5865: Status 404 returned error can't find the container with id 39e20f2343e756fc30c65087356964515a89bd86429bfbf7262d6cda5a8f5865
Jul 21 12:14:01.123193 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:01.123162    2562 generic.go:356] "Generic (PLEG): container finished" podID="1f1f12cb-ae83-409a-90ea-fa45b7f7434f" containerID="7272d340918b4dc942aed5d0b3555407dc139550b2f5ec87471b49727c7b2025" exitCode=0
Jul 21 12:14:01.123580 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:01.123254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f1f12cb-ae83-409a-90ea-fa45b7f7434f","Type":"ContainerDied","Data":"7272d340918b4dc942aed5d0b3555407dc139550b2f5ec87471b49727c7b2025"}
Jul 21 12:14:01.123580 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:01.123322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f1f12cb-ae83-409a-90ea-fa45b7f7434f","Type":"ContainerStarted","Data":"39e20f2343e756fc30c65087356964515a89bd86429bfbf7262d6cda5a8f5865"}
Jul 21 12:14:02.131507 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:02.131465    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f1f12cb-ae83-409a-90ea-fa45b7f7434f","Type":"ContainerStarted","Data":"185e98085c8c84ddaf6cf04ef6ea0877fccb6c7bac23a5dce2bad46a445da9bc"}
Jul 21 12:14:02.131951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:02.131532    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f1f12cb-ae83-409a-90ea-fa45b7f7434f","Type":"ContainerStarted","Data":"7b7e18b01834b010aedf430cff6f018ef497bd95e05e99d1779b5f5efb8075e1"}
Jul 21 12:14:02.131951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:02.131559    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f1f12cb-ae83-409a-90ea-fa45b7f7434f","Type":"ContainerStarted","Data":"5f3a069fe7691f64ae2bf81fcda08e948b01e3af099d566e8cac47f867d0e2eb"}
Jul 21 12:14:02.131951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:02.131587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f1f12cb-ae83-409a-90ea-fa45b7f7434f","Type":"ContainerStarted","Data":"6b770cad1bd88f58f2c46743ba10ca3260affd6915a2ee457a35080a469986ca"}
Jul 21 12:14:02.131951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:02.131615    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f1f12cb-ae83-409a-90ea-fa45b7f7434f","Type":"ContainerStarted","Data":"18a416e1f0c1e55905c5e0f256cb0b4a8e2b93947d882f67c52db8a84500f662"}
Jul 21 12:14:02.131951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:02.131638    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f1f12cb-ae83-409a-90ea-fa45b7f7434f","Type":"ContainerStarted","Data":"22e2d4dd87490428d0c65cee56062b92df5206cc093e4b8b64200107c5a0e69c"}
Jul 21 12:14:02.167287 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:02.167233    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.15110598 podStartE2EDuration="2.167215274s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.109294ms" isStatefulPod=false podCreationTimestamp="2026-07-21 12:14:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:14:02.165403363 +0000 UTC m=+166.223834366" watchObservedRunningTime="2026-07-21 12:14:02.167215274 +0000 UTC m=+166.225646280"
Jul 21 12:14:29.220538 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:29.220499    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74fd9659cc-m76jm"]
Jul 21 12:14:54.241159 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.241110    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74fd9659cc-m76jm" podUID="41c2be73-10c1-434e-8e51-fd74f138fd0f" containerName="console" containerID="cri-o://a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72" gracePeriod=15
Jul 21 12:14:54.477691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.477667    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:14:54.538182 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538110    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-config\") pod \"41c2be73-10c1-434e-8e51-fd74f138fd0f\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") "
Jul 21 12:14:54.538182 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538168    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-trusted-ca-bundle\") pod \"41c2be73-10c1-434e-8e51-fd74f138fd0f\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") "
Jul 21 12:14:54.538356 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538197    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-serving-cert\") pod \"41c2be73-10c1-434e-8e51-fd74f138fd0f\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") "
Jul 21 12:14:54.538356 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538226    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-oauth-serving-cert\") pod \"41c2be73-10c1-434e-8e51-fd74f138fd0f\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") "
Jul 21 12:14:54.538356 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538252    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-oauth-config\") pod \"41c2be73-10c1-434e-8e51-fd74f138fd0f\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") "
Jul 21 12:14:54.538356 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538279    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jh2n\" (UniqueName: \"kubernetes.io/projected/41c2be73-10c1-434e-8e51-fd74f138fd0f-kube-api-access-7jh2n\") pod \"41c2be73-10c1-434e-8e51-fd74f138fd0f\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") "
Jul 21 12:14:54.538356 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538331    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-service-ca\") pod \"41c2be73-10c1-434e-8e51-fd74f138fd0f\" (UID: \"41c2be73-10c1-434e-8e51-fd74f138fd0f\") "
Jul 21 12:14:54.538596 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538568    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-config" (OuterVolumeSpecName: "console-config") pod "41c2be73-10c1-434e-8e51-fd74f138fd0f" (UID: "41c2be73-10c1-434e-8e51-fd74f138fd0f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:14:54.538659 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538617    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:14:54.538659 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538625    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "41c2be73-10c1-434e-8e51-fd74f138fd0f" (UID: "41c2be73-10c1-434e-8e51-fd74f138fd0f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:14:54.538757 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.538664    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "41c2be73-10c1-434e-8e51-fd74f138fd0f" (UID: "41c2be73-10c1-434e-8e51-fd74f138fd0f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:14:54.539060 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.539034    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-service-ca" (OuterVolumeSpecName: "service-ca") pod "41c2be73-10c1-434e-8e51-fd74f138fd0f" (UID: "41c2be73-10c1-434e-8e51-fd74f138fd0f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:14:54.540717 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.540690    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41c2be73-10c1-434e-8e51-fd74f138fd0f-kube-api-access-7jh2n" (OuterVolumeSpecName: "kube-api-access-7jh2n") pod "41c2be73-10c1-434e-8e51-fd74f138fd0f" (UID: "41c2be73-10c1-434e-8e51-fd74f138fd0f"). InnerVolumeSpecName "kube-api-access-7jh2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:14:54.540978 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.540950    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "41c2be73-10c1-434e-8e51-fd74f138fd0f" (UID: "41c2be73-10c1-434e-8e51-fd74f138fd0f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:14:54.541065 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.540975    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "41c2be73-10c1-434e-8e51-fd74f138fd0f" (UID: "41c2be73-10c1-434e-8e51-fd74f138fd0f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:14:54.639854 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.639815    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-trusted-ca-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:14:54.639854 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.639845    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:14:54.639854 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.639855    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-oauth-serving-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:14:54.639854 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.639865    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c2be73-10c1-434e-8e51-fd74f138fd0f-console-oauth-config\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:14:54.640096 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.639873    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jh2n\" (UniqueName: \"kubernetes.io/projected/41c2be73-10c1-434e-8e51-fd74f138fd0f-kube-api-access-7jh2n\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:14:54.640096 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:54.639882    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c2be73-10c1-434e-8e51-fd74f138fd0f-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:14:55.322243 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:55.322204    2562 generic.go:356] "Generic (PLEG): container finished" podID="41c2be73-10c1-434e-8e51-fd74f138fd0f" containerID="a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72" exitCode=0
Jul 21 12:14:55.322622 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:55.322255    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74fd9659cc-m76jm"
Jul 21 12:14:55.322622 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:55.322290    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74fd9659cc-m76jm" event={"ID":"41c2be73-10c1-434e-8e51-fd74f138fd0f","Type":"ContainerDied","Data":"a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72"}
Jul 21 12:14:55.322622 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:55.322337    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74fd9659cc-m76jm" event={"ID":"41c2be73-10c1-434e-8e51-fd74f138fd0f","Type":"ContainerDied","Data":"4a1802c6cf5322f3110899990918a213a1c3bad1a640e94018d91825879ceda8"}
Jul 21 12:14:55.322622 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:55.322360    2562 scope.go:117] "RemoveContainer" containerID="a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72"
Jul 21 12:14:55.330636 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:55.330615    2562 scope.go:117] "RemoveContainer" containerID="a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72"
Jul 21 12:14:55.330920 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:14:55.330902    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72\": container with ID starting with a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72 not found: ID does not exist" containerID="a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72"
Jul 21 12:14:55.330974 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:55.330928    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72"} err="failed to get container status \"a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72\": rpc error: code = NotFound desc = could not find container \"a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72\": container with ID starting with a9a35e9e803fe5da86ba988a5abe4eb6c40f8dc1c271fd166ceabc8c7229ac72 not found: ID does not exist"
Jul 21 12:14:55.341973 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:55.341952    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74fd9659cc-m76jm"]
Jul 21 12:14:55.345610 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:55.345586    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74fd9659cc-m76jm"]
Jul 21 12:14:56.416298 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:14:56.416267    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41c2be73-10c1-434e-8e51-fd74f138fd0f" path="/var/lib/kubelet/pods/41c2be73-10c1-434e-8e51-fd74f138fd0f/volumes"
Jul 21 12:16:16.268613 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:16:16.268569    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 12:20:32.268644 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.268563    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"]
Jul 21 12:20:32.269108 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.268825    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41c2be73-10c1-434e-8e51-fd74f138fd0f" containerName="console"
Jul 21 12:20:32.269108 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.268837    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="41c2be73-10c1-434e-8e51-fd74f138fd0f" containerName="console"
Jul 21 12:20:32.269108 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.268890    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="41c2be73-10c1-434e-8e51-fd74f138fd0f" containerName="console"
Jul 21 12:20:32.271675 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.271659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.274265 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.274236    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 12:20:32.275188 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.275170    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 12:20:32.275254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.275212    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hndw5\""
Jul 21 12:20:32.290287 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.290260    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"]
Jul 21 12:20:32.364432 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.364395    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.364432 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.364433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.364656 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.364535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v2c5\" (UniqueName: \"kubernetes.io/projected/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-kube-api-access-4v2c5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.465631 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.465588    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v2c5\" (UniqueName: \"kubernetes.io/projected/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-kube-api-access-4v2c5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.465872 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.465649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.465872 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.465670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.466069 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.466050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.466113 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.466102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.477897 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.477873    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v2c5\" (UniqueName: \"kubernetes.io/projected/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-kube-api-access-4v2c5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.580650 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.580620    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:32.707383 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.707352    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"]
Jul 21 12:20:32.710453 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:20:32.710426    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a244e89_190a_43f6_ac6c_6bb49c3ad47e.slice/crio-4121ce451f74d840f4791921c068bcc626cdfae11c422fed3095bdfa6de2a6bd WatchSource:0}: Error finding container 4121ce451f74d840f4791921c068bcc626cdfae11c422fed3095bdfa6de2a6bd: Status 404 returned error can't find the container with id 4121ce451f74d840f4791921c068bcc626cdfae11c422fed3095bdfa6de2a6bd
Jul 21 12:20:32.712798 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:32.712765    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:20:33.433957 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:33.433922    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr" event={"ID":"7a244e89-190a-43f6-ac6c-6bb49c3ad47e","Type":"ContainerStarted","Data":"4121ce451f74d840f4791921c068bcc626cdfae11c422fed3095bdfa6de2a6bd"}
Jul 21 12:20:39.457492 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:39.457452    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerID="b57e4885ea56d4c62901932574db87c95893e468d18599336b86d4d47a8a6415" exitCode=0
Jul 21 12:20:39.457879 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:39.457528    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr" event={"ID":"7a244e89-190a-43f6-ac6c-6bb49c3ad47e","Type":"ContainerDied","Data":"b57e4885ea56d4c62901932574db87c95893e468d18599336b86d4d47a8a6415"}
Jul 21 12:20:41.465660 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:41.465621    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerID="e81d38126a20677795dfcda17578bef15dc1ce4c2d1882e4f50b8cc97d3c9f9f" exitCode=0
Jul 21 12:20:41.466040 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:41.465705    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr" event={"ID":"7a244e89-190a-43f6-ac6c-6bb49c3ad47e","Type":"ContainerDied","Data":"e81d38126a20677795dfcda17578bef15dc1ce4c2d1882e4f50b8cc97d3c9f9f"}
Jul 21 12:20:49.495023 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:49.494982    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerID="1bff49418845537c7176268b704e38ff6c23e85692d9d7ab1739a2434356890a" exitCode=0
Jul 21 12:20:49.495420 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:49.495075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr" event={"ID":"7a244e89-190a-43f6-ac6c-6bb49c3ad47e","Type":"ContainerDied","Data":"1bff49418845537c7176268b704e38ff6c23e85692d9d7ab1739a2434356890a"}
Jul 21 12:20:50.621831 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.621807    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:50.716640 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.716600    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v2c5\" (UniqueName: \"kubernetes.io/projected/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-kube-api-access-4v2c5\") pod \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") "
Jul 21 12:20:50.716640 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.716646    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-bundle\") pod \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") "
Jul 21 12:20:50.716897 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.716683    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-util\") pod \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\" (UID: \"7a244e89-190a-43f6-ac6c-6bb49c3ad47e\") "
Jul 21 12:20:50.717345 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.717320    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-bundle" (OuterVolumeSpecName: "bundle") pod "7a244e89-190a-43f6-ac6c-6bb49c3ad47e" (UID: "7a244e89-190a-43f6-ac6c-6bb49c3ad47e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:20:50.718966 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.718936    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-kube-api-access-4v2c5" (OuterVolumeSpecName: "kube-api-access-4v2c5") pod "7a244e89-190a-43f6-ac6c-6bb49c3ad47e" (UID: "7a244e89-190a-43f6-ac6c-6bb49c3ad47e"). InnerVolumeSpecName "kube-api-access-4v2c5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:20:50.722046 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.722021    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-util" (OuterVolumeSpecName: "util") pod "7a244e89-190a-43f6-ac6c-6bb49c3ad47e" (UID: "7a244e89-190a-43f6-ac6c-6bb49c3ad47e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:20:50.818170 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.818136    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v2c5\" (UniqueName: \"kubernetes.io/projected/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-kube-api-access-4v2c5\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:20:50.818170 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.818165    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:20:50.818170 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:50.818176    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a244e89-190a-43f6-ac6c-6bb49c3ad47e-util\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:20:51.505334 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:51.505298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr" event={"ID":"7a244e89-190a-43f6-ac6c-6bb49c3ad47e","Type":"ContainerDied","Data":"4121ce451f74d840f4791921c068bcc626cdfae11c422fed3095bdfa6de2a6bd"}
Jul 21 12:20:51.505498 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:51.505344    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4121ce451f74d840f4791921c068bcc626cdfae11c422fed3095bdfa6de2a6bd"
Jul 21 12:20:51.505498 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:51.505352    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xlqrr"
Jul 21 12:20:55.512053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.512020    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"]
Jul 21 12:20:55.512431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.512258    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerName="extract"
Jul 21 12:20:55.512431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.512268    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerName="extract"
Jul 21 12:20:55.512431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.512284    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerName="pull"
Jul 21 12:20:55.512431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.512289    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerName="pull"
Jul 21 12:20:55.512431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.512310    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerName="util"
Jul 21 12:20:55.512431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.512316    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerName="util"
Jul 21 12:20:55.512431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.512355    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a244e89-190a-43f6-ac6c-6bb49c3ad47e" containerName="extract"
Jul 21 12:20:55.558354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.558325    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"]
Jul 21 12:20:55.558515 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.558417    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"
Jul 21 12:20:55.561021 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.560997    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-hjj4t\""
Jul 21 12:20:55.561354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.561335    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 21 12:20:55.561863 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.561849    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:20:55.655897 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.655858    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zh2q\" (UniqueName: \"kubernetes.io/projected/ce37544d-d616-4af0-88f7-1c796350983b-kube-api-access-2zh2q\") pod \"cert-manager-operator-controller-manager-55c444bf86-j9n5n\" (UID: \"ce37544d-d616-4af0-88f7-1c796350983b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"
Jul 21 12:20:55.656064 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.655903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce37544d-d616-4af0-88f7-1c796350983b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-j9n5n\" (UID: \"ce37544d-d616-4af0-88f7-1c796350983b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"
Jul 21 12:20:55.756707 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.756670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zh2q\" (UniqueName: \"kubernetes.io/projected/ce37544d-d616-4af0-88f7-1c796350983b-kube-api-access-2zh2q\") pod \"cert-manager-operator-controller-manager-55c444bf86-j9n5n\" (UID: \"ce37544d-d616-4af0-88f7-1c796350983b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"
Jul 21 12:20:55.756707 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.756708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce37544d-d616-4af0-88f7-1c796350983b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-j9n5n\" (UID: \"ce37544d-d616-4af0-88f7-1c796350983b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"
Jul 21 12:20:55.757053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.757036    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce37544d-d616-4af0-88f7-1c796350983b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-j9n5n\" (UID: \"ce37544d-d616-4af0-88f7-1c796350983b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"
Jul 21 12:20:55.769147 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.769073    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zh2q\" (UniqueName: \"kubernetes.io/projected/ce37544d-d616-4af0-88f7-1c796350983b-kube-api-access-2zh2q\") pod \"cert-manager-operator-controller-manager-55c444bf86-j9n5n\" (UID: \"ce37544d-d616-4af0-88f7-1c796350983b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"
Jul 21 12:20:55.867727 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:55.867689    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"
Jul 21 12:20:56.031377 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:56.031303    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n"]
Jul 21 12:20:56.034971 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:20:56.034945    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce37544d_d616_4af0_88f7_1c796350983b.slice/crio-8469fee5a3150920e7365a775a94d630899bc095a40d7035e7c7333e0cc929fc WatchSource:0}: Error finding container 8469fee5a3150920e7365a775a94d630899bc095a40d7035e7c7333e0cc929fc: Status 404 returned error can't find the container with id 8469fee5a3150920e7365a775a94d630899bc095a40d7035e7c7333e0cc929fc
Jul 21 12:20:56.522853 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:56.522814    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n" event={"ID":"ce37544d-d616-4af0-88f7-1c796350983b","Type":"ContainerStarted","Data":"8469fee5a3150920e7365a775a94d630899bc095a40d7035e7c7333e0cc929fc"}
Jul 21 12:20:58.533014 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:58.532925    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n" event={"ID":"ce37544d-d616-4af0-88f7-1c796350983b","Type":"ContainerStarted","Data":"d44153f570c854922130d90c19615a5e137e2a59da968dbe5dcdc05b5978f8f4"}
Jul 21 12:20:58.556103 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:20:58.556027    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-j9n5n" podStartSLOduration=1.340058378 podStartE2EDuration="3.556013817s" totalImagesPullingTime="2.215955439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:20:58.554947007 +0000 UTC m=+582.613378024" watchObservedRunningTime="2026-07-21 12:20:58.556013817 +0000 UTC m=+582.614444820"
Jul 21 12:21:01.117057 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.117019    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"]
Jul 21 12:21:01.119663 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.119643    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.122286 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.122261    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 12:21:01.122439 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.122376    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hndw5\""
Jul 21 12:21:01.123115 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.123091    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 12:21:01.130830 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.130808    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"]
Jul 21 12:21:01.196750 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.196715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.196929 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.196790    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.196929 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.196840    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcjg6\" (UniqueName: \"kubernetes.io/projected/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-kube-api-access-fcjg6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.297331 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.297297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.297331 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.297333    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjg6\" (UniqueName: \"kubernetes.io/projected/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-kube-api-access-fcjg6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.297567 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.297381    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.297732 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.297708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.297832 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.297743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.309230 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.309199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcjg6\" (UniqueName: \"kubernetes.io/projected/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-kube-api-access-fcjg6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.428937 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.428847    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:01.562427 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:01.562400    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"]
Jul 21 12:21:01.565316 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:21:01.565290    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a0be6bd_3db4_4200_bca0_af66c2bd42bb.slice/crio-a47fd440389534522810aaed330450da308d34a0d299868819f4e1d88360dba9 WatchSource:0}: Error finding container a47fd440389534522810aaed330450da308d34a0d299868819f4e1d88360dba9: Status 404 returned error can't find the container with id a47fd440389534522810aaed330450da308d34a0d299868819f4e1d88360dba9
Jul 21 12:21:02.547629 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:02.547597    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerID="2fe1a695cad4287f05995cc51961b60a3f27bbfd57f7e3fd3a7bb34b49b6584f" exitCode=0
Jul 21 12:21:02.548045 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:02.547662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47" event={"ID":"7a0be6bd-3db4-4200-bca0-af66c2bd42bb","Type":"ContainerDied","Data":"2fe1a695cad4287f05995cc51961b60a3f27bbfd57f7e3fd3a7bb34b49b6584f"}
Jul 21 12:21:02.548045 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:02.547698    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47" event={"ID":"7a0be6bd-3db4-4200-bca0-af66c2bd42bb","Type":"ContainerStarted","Data":"a47fd440389534522810aaed330450da308d34a0d299868819f4e1d88360dba9"}
Jul 21 12:21:05.560989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:05.560956    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerID="4496cab5a36e1ea622b147c743321f1ed778813723e56e85d7fd806469855294" exitCode=0
Jul 21 12:21:05.560989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:05.560992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47" event={"ID":"7a0be6bd-3db4-4200-bca0-af66c2bd42bb","Type":"ContainerDied","Data":"4496cab5a36e1ea622b147c743321f1ed778813723e56e85d7fd806469855294"}
Jul 21 12:21:06.566343 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:06.566305    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerID="366fe926813fcbb7197a9b77d2029b80308ba2a7245bfbdba3325d25c89ab7cc" exitCode=0
Jul 21 12:21:06.566704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:06.566348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47" event={"ID":"7a0be6bd-3db4-4200-bca0-af66c2bd42bb","Type":"ContainerDied","Data":"366fe926813fcbb7197a9b77d2029b80308ba2a7245bfbdba3325d25c89ab7cc"}
Jul 21 12:21:07.693842 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.693817    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:07.849804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.849691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-bundle\") pod \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") "
Jul 21 12:21:07.849804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.849794    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcjg6\" (UniqueName: \"kubernetes.io/projected/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-kube-api-access-fcjg6\") pod \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") "
Jul 21 12:21:07.849997 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.849826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-util\") pod \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\" (UID: \"7a0be6bd-3db4-4200-bca0-af66c2bd42bb\") "
Jul 21 12:21:07.850248 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.850222    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-bundle" (OuterVolumeSpecName: "bundle") pod "7a0be6bd-3db4-4200-bca0-af66c2bd42bb" (UID: "7a0be6bd-3db4-4200-bca0-af66c2bd42bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:21:07.852029 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.852006    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-kube-api-access-fcjg6" (OuterVolumeSpecName: "kube-api-access-fcjg6") pod "7a0be6bd-3db4-4200-bca0-af66c2bd42bb" (UID: "7a0be6bd-3db4-4200-bca0-af66c2bd42bb"). InnerVolumeSpecName "kube-api-access-fcjg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:21:07.854448 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.854428    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-util" (OuterVolumeSpecName: "util") pod "7a0be6bd-3db4-4200-bca0-af66c2bd42bb" (UID: "7a0be6bd-3db4-4200-bca0-af66c2bd42bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:21:07.950547 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.950507    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:21:07.950547 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.950543    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcjg6\" (UniqueName: \"kubernetes.io/projected/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-kube-api-access-fcjg6\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:21:07.950547 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:07.950553    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a0be6bd-3db4-4200-bca0-af66c2bd42bb-util\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:21:08.575759 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:08.575731    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47"
Jul 21 12:21:08.575930 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:08.575731    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frzf47" event={"ID":"7a0be6bd-3db4-4200-bca0-af66c2bd42bb","Type":"ContainerDied","Data":"a47fd440389534522810aaed330450da308d34a0d299868819f4e1d88360dba9"}
Jul 21 12:21:08.575930 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:08.575828    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a47fd440389534522810aaed330450da308d34a0d299868819f4e1d88360dba9"
Jul 21 12:21:10.206586 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.206545    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-465bb"]
Jul 21 12:21:10.207091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.206916    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerName="extract"
Jul 21 12:21:10.207091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.206936    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerName="extract"
Jul 21 12:21:10.207091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.206962    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerName="util"
Jul 21 12:21:10.207091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.206970    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerName="util"
Jul 21 12:21:10.207091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.206982    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerName="pull"
Jul 21 12:21:10.207091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.206989    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerName="pull"
Jul 21 12:21:10.207091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.207050    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a0be6bd-3db4-4200-bca0-af66c2bd42bb" containerName="extract"
Jul 21 12:21:10.209929 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.209909    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-465bb"
Jul 21 12:21:10.212469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.212445    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 21 12:21:10.212557 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.212445    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ck8v7\""
Jul 21 12:21:10.213347 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.213326    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 21 12:21:10.220513 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.220491    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-465bb"]
Jul 21 12:21:10.367745 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.367702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc8t7\" (UniqueName: \"kubernetes.io/projected/101ca2bf-4f86-4d29-8bea-192d56e141d2-kube-api-access-xc8t7\") pod \"cert-manager-55b4d954d5-465bb\" (UID: \"101ca2bf-4f86-4d29-8bea-192d56e141d2\") " pod="cert-manager/cert-manager-55b4d954d5-465bb"
Jul 21 12:21:10.367966 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.367853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/101ca2bf-4f86-4d29-8bea-192d56e141d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-465bb\" (UID: \"101ca2bf-4f86-4d29-8bea-192d56e141d2\") " pod="cert-manager/cert-manager-55b4d954d5-465bb"
Jul 21 12:21:10.469056 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.468978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/101ca2bf-4f86-4d29-8bea-192d56e141d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-465bb\" (UID: \"101ca2bf-4f86-4d29-8bea-192d56e141d2\") " pod="cert-manager/cert-manager-55b4d954d5-465bb"
Jul 21 12:21:10.469056 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.469019    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc8t7\" (UniqueName: \"kubernetes.io/projected/101ca2bf-4f86-4d29-8bea-192d56e141d2-kube-api-access-xc8t7\") pod \"cert-manager-55b4d954d5-465bb\" (UID: \"101ca2bf-4f86-4d29-8bea-192d56e141d2\") " pod="cert-manager/cert-manager-55b4d954d5-465bb"
Jul 21 12:21:10.477650 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.477623    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/101ca2bf-4f86-4d29-8bea-192d56e141d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-465bb\" (UID: \"101ca2bf-4f86-4d29-8bea-192d56e141d2\") " pod="cert-manager/cert-manager-55b4d954d5-465bb"
Jul 21 12:21:10.477769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.477731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc8t7\" (UniqueName: \"kubernetes.io/projected/101ca2bf-4f86-4d29-8bea-192d56e141d2-kube-api-access-xc8t7\") pod \"cert-manager-55b4d954d5-465bb\" (UID: \"101ca2bf-4f86-4d29-8bea-192d56e141d2\") " pod="cert-manager/cert-manager-55b4d954d5-465bb"
Jul 21 12:21:10.518642 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.518614    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-465bb"
Jul 21 12:21:10.641689 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:10.641645    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-465bb"]
Jul 21 12:21:10.644484 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:21:10.644453    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod101ca2bf_4f86_4d29_8bea_192d56e141d2.slice/crio-d15a4f7fc35e340ee0aedaf7781605e9ee224b6395120db3afb010aadda8ff90 WatchSource:0}: Error finding container d15a4f7fc35e340ee0aedaf7781605e9ee224b6395120db3afb010aadda8ff90: Status 404 returned error can't find the container with id d15a4f7fc35e340ee0aedaf7781605e9ee224b6395120db3afb010aadda8ff90
Jul 21 12:21:11.594044 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:11.594003    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-465bb" event={"ID":"101ca2bf-4f86-4d29-8bea-192d56e141d2","Type":"ContainerStarted","Data":"d15a4f7fc35e340ee0aedaf7781605e9ee224b6395120db3afb010aadda8ff90"}
Jul 21 12:21:13.604239 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:13.604206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-465bb" event={"ID":"101ca2bf-4f86-4d29-8bea-192d56e141d2","Type":"ContainerStarted","Data":"03f8a12a9d2aef0d192c996a8e9277ae93403ba8ad1b2dd2b31cb997d84092e1"}
Jul 21 12:21:13.624210 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:13.624110    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-465bb" podStartSLOduration=0.91283966 podStartE2EDuration="3.624091296s" totalImagesPullingTime="2.711251636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:21:13.623418901 +0000 UTC m=+597.681849904" watchObservedRunningTime="2026-07-21 12:21:13.624091296 +0000 UTC m=+597.682522302"
Jul 21 12:21:20.523075 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.523034    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"]
Jul 21 12:21:20.527254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.527229    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.530405 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.530379    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hndw5\""
Jul 21 12:21:20.530819 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.530806    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 12:21:20.531333 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.531318    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 12:21:20.540042 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.540015    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"]
Jul 21 12:21:20.545769 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.545744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.545872 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.545822    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.545929 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.545895    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6tmk\" (UniqueName: \"kubernetes.io/projected/e2d48968-dabb-47bc-b388-103290339aab-kube-api-access-x6tmk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.646824 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.646788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6tmk\" (UniqueName: \"kubernetes.io/projected/e2d48968-dabb-47bc-b388-103290339aab-kube-api-access-x6tmk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.646985 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.646858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.646985 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.646889    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.647245 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.647228    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.647285 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.647259    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.657881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.657848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6tmk\" (UniqueName: \"kubernetes.io/projected/e2d48968-dabb-47bc-b388-103290339aab-kube-api-access-x6tmk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.836618 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.836575    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:20.974667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:20.974620    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"]
Jul 21 12:21:20.978729 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:21:20.978703    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d48968_dabb_47bc_b388_103290339aab.slice/crio-f0b7663378609db546866cbc61f31ce0cbe3c18007db3b6ac5553111aea5473e WatchSource:0}: Error finding container f0b7663378609db546866cbc61f31ce0cbe3c18007db3b6ac5553111aea5473e: Status 404 returned error can't find the container with id f0b7663378609db546866cbc61f31ce0cbe3c18007db3b6ac5553111aea5473e
Jul 21 12:21:21.637687 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:21.637648    2562 generic.go:356] "Generic (PLEG): container finished" podID="e2d48968-dabb-47bc-b388-103290339aab" containerID="058ac1c3f96e0fd81e02b1550b6fbbf7a7554fcaf221137d5cfdbd264faf214f" exitCode=0
Jul 21 12:21:21.638084 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:21.637725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b" event={"ID":"e2d48968-dabb-47bc-b388-103290339aab","Type":"ContainerDied","Data":"058ac1c3f96e0fd81e02b1550b6fbbf7a7554fcaf221137d5cfdbd264faf214f"}
Jul 21 12:21:21.638084 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:21.637769    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b" event={"ID":"e2d48968-dabb-47bc-b388-103290339aab","Type":"ContainerStarted","Data":"f0b7663378609db546866cbc61f31ce0cbe3c18007db3b6ac5553111aea5473e"}
Jul 21 12:21:22.643846 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:22.643746    2562 generic.go:356] "Generic (PLEG): container finished" podID="e2d48968-dabb-47bc-b388-103290339aab" containerID="139aa54c135fe279190e4634f68d1ef05ea4b418247dc2920b097c5dada8953b" exitCode=0
Jul 21 12:21:22.643846 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:22.643820    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b" event={"ID":"e2d48968-dabb-47bc-b388-103290339aab","Type":"ContainerDied","Data":"139aa54c135fe279190e4634f68d1ef05ea4b418247dc2920b097c5dada8953b"}
Jul 21 12:21:23.649970 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:23.649936    2562 generic.go:356] "Generic (PLEG): container finished" podID="e2d48968-dabb-47bc-b388-103290339aab" containerID="2f7789a0d42e630a5d0b246c2bb76736f0e1cef302dd338517c7198138840243" exitCode=0
Jul 21 12:21:23.650330 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:23.650006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b" event={"ID":"e2d48968-dabb-47bc-b388-103290339aab","Type":"ContainerDied","Data":"2f7789a0d42e630a5d0b246c2bb76736f0e1cef302dd338517c7198138840243"}
Jul 21 12:21:24.782139 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.782109    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:24.875028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.874992    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-util\") pod \"e2d48968-dabb-47bc-b388-103290339aab\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") "
Jul 21 12:21:24.875028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.875028    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6tmk\" (UniqueName: \"kubernetes.io/projected/e2d48968-dabb-47bc-b388-103290339aab-kube-api-access-x6tmk\") pod \"e2d48968-dabb-47bc-b388-103290339aab\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") "
Jul 21 12:21:24.875216 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.875058    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-bundle\") pod \"e2d48968-dabb-47bc-b388-103290339aab\" (UID: \"e2d48968-dabb-47bc-b388-103290339aab\") "
Jul 21 12:21:24.875785 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.875745    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-bundle" (OuterVolumeSpecName: "bundle") pod "e2d48968-dabb-47bc-b388-103290339aab" (UID: "e2d48968-dabb-47bc-b388-103290339aab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:21:24.877228 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.877204    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d48968-dabb-47bc-b388-103290339aab-kube-api-access-x6tmk" (OuterVolumeSpecName: "kube-api-access-x6tmk") pod "e2d48968-dabb-47bc-b388-103290339aab" (UID: "e2d48968-dabb-47bc-b388-103290339aab"). InnerVolumeSpecName "kube-api-access-x6tmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:21:24.883214 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.883176    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-util" (OuterVolumeSpecName: "util") pod "e2d48968-dabb-47bc-b388-103290339aab" (UID: "e2d48968-dabb-47bc-b388-103290339aab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:21:24.975867 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.975749    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-util\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:21:24.975867 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.975817    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6tmk\" (UniqueName: \"kubernetes.io/projected/e2d48968-dabb-47bc-b388-103290339aab-kube-api-access-x6tmk\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:21:24.975867 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:24.975828    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2d48968-dabb-47bc-b388-103290339aab-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:21:25.659762 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:25.659723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b" event={"ID":"e2d48968-dabb-47bc-b388-103290339aab","Type":"ContainerDied","Data":"f0b7663378609db546866cbc61f31ce0cbe3c18007db3b6ac5553111aea5473e"}
Jul 21 12:21:25.659762 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:25.659752    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afl867b"
Jul 21 12:21:25.660054 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:25.659795    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0b7663378609db546866cbc61f31ce0cbe3c18007db3b6ac5553111aea5473e"
Jul 21 12:21:33.501299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.501259    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"]
Jul 21 12:21:33.501698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.501533    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2d48968-dabb-47bc-b388-103290339aab" containerName="extract"
Jul 21 12:21:33.501698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.501549    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2d48968-dabb-47bc-b388-103290339aab" containerName="extract"
Jul 21 12:21:33.501698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.501566    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2d48968-dabb-47bc-b388-103290339aab" containerName="pull"
Jul 21 12:21:33.501698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.501572    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2d48968-dabb-47bc-b388-103290339aab" containerName="pull"
Jul 21 12:21:33.501698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.501583    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2d48968-dabb-47bc-b388-103290339aab" containerName="util"
Jul 21 12:21:33.501698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.501588    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2d48968-dabb-47bc-b388-103290339aab" containerName="util"
Jul 21 12:21:33.501698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.501632    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2d48968-dabb-47bc-b388-103290339aab" containerName="extract"
Jul 21 12:21:33.507852 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.507753    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.511157 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.511138    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 12:21:33.511749 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.511720    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 12:21:33.511878 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.511802    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 12:21:33.512053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.512037    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b5mqv\""
Jul 21 12:21:33.512135 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.512044    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 12:21:33.522021 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.522000    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"]
Jul 21 12:21:33.541271 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.541240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-tmp\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.541384 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.541285    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-webhook-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.541384 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.541338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n95j\" (UniqueName: \"kubernetes.io/projected/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-kube-api-access-9n95j\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.541384 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.541365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-manifests\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.541487 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.541386    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.642278 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.642244    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-tmp\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.642445 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.642289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-webhook-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.642445 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.642404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n95j\" (UniqueName: \"kubernetes.io/projected/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-kube-api-access-9n95j\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.642540 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.642452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-manifests\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.642540 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.642471    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.642899 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.642875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-manifests\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.644761 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.644741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-tmp\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.644989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.644968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.645037 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.644979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-webhook-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.652955 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.652935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n95j\" (UniqueName: \"kubernetes.io/projected/7bd1806e-5e0f-4baa-b3d5-35776ff78d94-kube-api-access-9n95j\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-gmbzq\" (UID: \"7bd1806e-5e0f-4baa-b3d5-35776ff78d94\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.817340 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.817308    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:33.977040 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:33.977017    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"]
Jul 21 12:21:33.979264 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:21:33.979240    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd1806e_5e0f_4baa_b3d5_35776ff78d94.slice/crio-d71673b02af840be30f9de6b57006de4b1954cb52b2deb85fd8f1280b6be3b5b WatchSource:0}: Error finding container d71673b02af840be30f9de6b57006de4b1954cb52b2deb85fd8f1280b6be3b5b: Status 404 returned error can't find the container with id d71673b02af840be30f9de6b57006de4b1954cb52b2deb85fd8f1280b6be3b5b
Jul 21 12:21:34.696459 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:34.696421    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq" event={"ID":"7bd1806e-5e0f-4baa-b3d5-35776ff78d94","Type":"ContainerStarted","Data":"d71673b02af840be30f9de6b57006de4b1954cb52b2deb85fd8f1280b6be3b5b"}
Jul 21 12:21:37.712368 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:37.712278    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq" event={"ID":"7bd1806e-5e0f-4baa-b3d5-35776ff78d94","Type":"ContainerStarted","Data":"8315a563427c92c86e4e127e9cd0a99cd757e0f28bd3d499e6d31b5bac277147"}
Jul 21 12:21:38.718249 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:38.718165    2562 generic.go:356] "Generic (PLEG): container finished" podID="7bd1806e-5e0f-4baa-b3d5-35776ff78d94" containerID="8315a563427c92c86e4e127e9cd0a99cd757e0f28bd3d499e6d31b5bac277147" exitCode=0
Jul 21 12:21:38.718249 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:38.718241    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq" event={"ID":"7bd1806e-5e0f-4baa-b3d5-35776ff78d94","Type":"ContainerDied","Data":"8315a563427c92c86e4e127e9cd0a99cd757e0f28bd3d499e6d31b5bac277147"}
Jul 21 12:21:39.724629 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:39.724587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq" event={"ID":"7bd1806e-5e0f-4baa-b3d5-35776ff78d94","Type":"ContainerStarted","Data":"3ee35e2808e27004f6651d0711b8485fec21b2465d7847ad7407bc6de74fe5e1"}
Jul 21 12:21:39.725020 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:39.724728    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:39.769007 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:39.768952    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq" podStartSLOduration=1.931978784 podStartE2EDuration="6.768937252s" totalImagesPullingTime="3.797934604s" totalInitContainerRuntime="1.039023864s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:21:39.766764815 +0000 UTC m=+623.825195819" watchObservedRunningTime="2026-07-21 12:21:39.768937252 +0000 UTC m=+623.827368255"
Jul 21 12:21:50.731249 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:50.731205    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-gmbzq"
Jul 21 12:21:51.018981 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.018906    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"]
Jul 21 12:21:51.025945 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.025920    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.028444 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.028399    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:21:51.028583 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.028473    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 21 12:21:51.028583 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.028524    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 21 12:21:51.028704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.028607    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 21 12:21:51.028795 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.028762    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vcw84\""
Jul 21 12:21:51.029320 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.029308    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 21 12:21:51.036248 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.036218    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"]
Jul 21 12:21:51.099123 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.099091    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/21e68d0a-af7e-4183-82d7-c5b1843de9ac-metrics-cert\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.099249 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.099155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67cmr\" (UniqueName: \"kubernetes.io/projected/21e68d0a-af7e-4183-82d7-c5b1843de9ac-kube-api-access-67cmr\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.099249 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.099202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/21e68d0a-af7e-4183-82d7-c5b1843de9ac-manager-config\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.099249 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.099232    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21e68d0a-af7e-4183-82d7-c5b1843de9ac-cert\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.200527 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.200484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67cmr\" (UniqueName: \"kubernetes.io/projected/21e68d0a-af7e-4183-82d7-c5b1843de9ac-kube-api-access-67cmr\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.200527 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.200540    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/21e68d0a-af7e-4183-82d7-c5b1843de9ac-manager-config\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.200747 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.200563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21e68d0a-af7e-4183-82d7-c5b1843de9ac-cert\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.200747 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.200595    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/21e68d0a-af7e-4183-82d7-c5b1843de9ac-metrics-cert\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.201279 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.201256    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/21e68d0a-af7e-4183-82d7-c5b1843de9ac-manager-config\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.203283 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.203264    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/21e68d0a-af7e-4183-82d7-c5b1843de9ac-metrics-cert\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.203341 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.203318    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21e68d0a-af7e-4183-82d7-c5b1843de9ac-cert\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.209855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.209834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67cmr\" (UniqueName: \"kubernetes.io/projected/21e68d0a-af7e-4183-82d7-c5b1843de9ac-kube-api-access-67cmr\") pod \"lws-controller-manager-c65cd9d66-jmsxk\" (UID: \"21e68d0a-af7e-4183-82d7-c5b1843de9ac\") " pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.335017 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.334988    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:51.464900 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.464872    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"]
Jul 21 12:21:51.467856 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:21:51.467827    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e68d0a_af7e_4183_82d7_c5b1843de9ac.slice/crio-114e0405c5e8bae4c416c8e1f5e6f601ae46cd691c55be07f508ad8ebff53e94 WatchSource:0}: Error finding container 114e0405c5e8bae4c416c8e1f5e6f601ae46cd691c55be07f508ad8ebff53e94: Status 404 returned error can't find the container with id 114e0405c5e8bae4c416c8e1f5e6f601ae46cd691c55be07f508ad8ebff53e94
Jul 21 12:21:51.775989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:51.775903    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk" event={"ID":"21e68d0a-af7e-4183-82d7-c5b1843de9ac","Type":"ContainerStarted","Data":"114e0405c5e8bae4c416c8e1f5e6f601ae46cd691c55be07f508ad8ebff53e94"}
Jul 21 12:21:53.787613 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:53.787576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk" event={"ID":"21e68d0a-af7e-4183-82d7-c5b1843de9ac","Type":"ContainerStarted","Data":"26d5ad062adfd1a2aa1c0fdffff754399b139a2e6b5df8a3370936b6af61c1d1"}
Jul 21 12:21:53.788022 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:53.787709    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:21:53.888750 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:21:53.888684    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk" podStartSLOduration=1.362511357 podStartE2EDuration="2.888662586s" totalImagesPullingTime="1.526151229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:21:53.874297148 +0000 UTC m=+637.932728151" watchObservedRunningTime="2026-07-21 12:21:53.888662586 +0000 UTC m=+637.947093592"
Jul 21 12:22:02.379075 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.379031    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"]
Jul 21 12:22:02.382380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.382359    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.394642 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.394616    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 12:22:02.394642 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.394638    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 12:22:02.394859 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.394638    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-k2sph\""
Jul 21 12:22:02.394859 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.394642    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 12:22:02.395562 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.395546    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 12:22:02.395653 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.395604    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 12:22:02.395653 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.395623    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 12:22:02.482239 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.482206    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"]
Jul 21 12:22:02.489250 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.489224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.489398 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.489266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.489398 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.489287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr58t\" (UniqueName: \"kubernetes.io/projected/a86f1e20-097a-4546-8ee8-232bb2c7c776-kube-api-access-dr58t\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.489398 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.489314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a86f1e20-097a-4546-8ee8-232bb2c7c776-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.489398 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.489395    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.489531 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.489443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a86f1e20-097a-4546-8ee8-232bb2c7c776-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.489531 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.489478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.590744 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.590708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.590744 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.590745    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a86f1e20-097a-4546-8ee8-232bb2c7c776-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.591048 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.590763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.591048 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.590813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.591048 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.590848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.591048 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.590873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr58t\" (UniqueName: \"kubernetes.io/projected/a86f1e20-097a-4546-8ee8-232bb2c7c776-kube-api-access-dr58t\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.591224 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.591118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a86f1e20-097a-4546-8ee8-232bb2c7c776-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.591574 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.591547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.593532 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.593506    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a86f1e20-097a-4546-8ee8-232bb2c7c776-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.593619 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.593510    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a86f1e20-097a-4546-8ee8-232bb2c7c776-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.593619 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.593591    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.593694 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.593601    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.609114 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.609091    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr58t\" (UniqueName: \"kubernetes.io/projected/a86f1e20-097a-4546-8ee8-232bb2c7c776-kube-api-access-dr58t\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.609674 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.609653    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a86f1e20-097a-4546-8ee8-232bb2c7c776-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pfllr\" (UID: \"a86f1e20-097a-4546-8ee8-232bb2c7c776\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.691416 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.691340    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:02.835393 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:02.835361    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"]
Jul 21 12:22:02.839021 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:22:02.838991    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda86f1e20_097a_4546_8ee8_232bb2c7c776.slice/crio-881a1fb9b48fd554a2351864d223be7a6b5338aed86adfdefd70c696f6894159 WatchSource:0}: Error finding container 881a1fb9b48fd554a2351864d223be7a6b5338aed86adfdefd70c696f6894159: Status 404 returned error can't find the container with id 881a1fb9b48fd554a2351864d223be7a6b5338aed86adfdefd70c696f6894159
Jul 21 12:22:03.828950 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:03.828917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr" event={"ID":"a86f1e20-097a-4546-8ee8-232bb2c7c776","Type":"ContainerStarted","Data":"881a1fb9b48fd554a2351864d223be7a6b5338aed86adfdefd70c696f6894159"}
Jul 21 12:22:04.795142 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:04.795108    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c65cd9d66-jmsxk"
Jul 21 12:22:06.522641 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:06.522599    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:22:06.523046 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:06.522682    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:22:06.845926 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:06.845892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr" event={"ID":"a86f1e20-097a-4546-8ee8-232bb2c7c776","Type":"ContainerStarted","Data":"f51b6a9e5ecb8fc3837c9d3d4286cdc342c3993f825b614b78d70f6d39338e8e"}
Jul 21 12:22:06.846104 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:06.846039    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:22:06.886052 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:06.885997    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr" podStartSLOduration=1.204342275 podStartE2EDuration="4.885977619s" totalImagesPullingTime="3.681635344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:22:06.883103484 +0000 UTC m=+650.941534500" watchObservedRunningTime="2026-07-21 12:22:06.885977619 +0000 UTC m=+650.944408623"
Jul 21 12:22:07.851699 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:07.851665    2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-pfllr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 12:22:07.852086 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:07.851722    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr" podUID="a86f1e20-097a-4546-8ee8-232bb2c7c776" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 12:22:10.851305 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:22:10.851275    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pfllr"
Jul 21 12:23:06.553411 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.553376    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"]
Jul 21 12:23:06.558156 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.558138    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.560555 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.560532    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 12:23:06.560676 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.560539    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 12:23:06.561489 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.561472    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mslb2\""
Jul 21 12:23:06.569051 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.569029    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"]
Jul 21 12:23:06.694729 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.694693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.694947 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.694810    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptw8l\" (UniqueName: \"kubernetes.io/projected/789e13b0-0f43-4814-8311-95da61ce53f1-kube-api-access-ptw8l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.694947 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.694851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.795663 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.795608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptw8l\" (UniqueName: \"kubernetes.io/projected/789e13b0-0f43-4814-8311-95da61ce53f1-kube-api-access-ptw8l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.795901 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.795676    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.795901 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.795717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.796159 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.796135    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.796225 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.796147    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.804428 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.804368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptw8l\" (UniqueName: \"kubernetes.io/projected/789e13b0-0f43-4814-8311-95da61ce53f1-kube-api-access-ptw8l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:06.867389 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:06.867349    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:07.001137 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.001110    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"]
Jul 21 12:23:07.003462 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:23:07.003430    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod789e13b0_0f43_4814_8311_95da61ce53f1.slice/crio-cfd399fea46090800eedc0a1c49aa79c77d81a99d21aa41a5e994de5171d4c4f WatchSource:0}: Error finding container cfd399fea46090800eedc0a1c49aa79c77d81a99d21aa41a5e994de5171d4c4f: Status 404 returned error can't find the container with id cfd399fea46090800eedc0a1c49aa79c77d81a99d21aa41a5e994de5171d4c4f
Jul 21 12:23:07.093305 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.093269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh" event={"ID":"789e13b0-0f43-4814-8311-95da61ce53f1","Type":"ContainerStarted","Data":"de0aa2bea6d3b0badba704a2a3343779207f1ee70fa3249432d6c816f9858c52"}
Jul 21 12:23:07.093445 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.093322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh" event={"ID":"789e13b0-0f43-4814-8311-95da61ce53f1","Type":"ContainerStarted","Data":"cfd399fea46090800eedc0a1c49aa79c77d81a99d21aa41a5e994de5171d4c4f"}
Jul 21 12:23:07.114230 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.114203    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"]
Jul 21 12:23:07.117513 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.117495    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.125648 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.125620    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"]
Jul 21 12:23:07.198884 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.198843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.198884 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.198884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52fv6\" (UniqueName: \"kubernetes.io/projected/56f76b3b-58fe-402b-9b02-938436473fc7-kube-api-access-52fv6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.199117 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.199053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.299542 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.299484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.299749 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.299582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.299749 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.299611    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fv6\" (UniqueName: \"kubernetes.io/projected/56f76b3b-58fe-402b-9b02-938436473fc7-kube-api-access-52fv6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.300033 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.300005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.300089 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.300005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.308011 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.307935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52fv6\" (UniqueName: \"kubernetes.io/projected/56f76b3b-58fe-402b-9b02-938436473fc7-kube-api-access-52fv6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.461150 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.461102    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:07.591720 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.591694    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"]
Jul 21 12:23:07.593166 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:23:07.593130    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56f76b3b_58fe_402b_9b02_938436473fc7.slice/crio-f6d3fc2047aba782c988032e1ed308afa58939759024b5371139c5489da49616 WatchSource:0}: Error finding container f6d3fc2047aba782c988032e1ed308afa58939759024b5371139c5489da49616: Status 404 returned error can't find the container with id f6d3fc2047aba782c988032e1ed308afa58939759024b5371139c5489da49616
Jul 21 12:23:07.710831 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.710803    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"]
Jul 21 12:23:07.714055 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.714038    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:07.722113 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.722090    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"]
Jul 21 12:23:07.803732 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.803693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lbvq\" (UniqueName: \"kubernetes.io/projected/3c45448a-d2cb-4b9e-8588-9c6324577b8b-kube-api-access-9lbvq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:07.803732 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.803738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:07.803966 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.803796    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:07.904303 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.904209    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lbvq\" (UniqueName: \"kubernetes.io/projected/3c45448a-d2cb-4b9e-8588-9c6324577b8b-kube-api-access-9lbvq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:07.904303 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.904257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:07.904303 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.904294    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:07.904692 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.904670    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:07.904734 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.904695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:07.913143 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:07.913107    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lbvq\" (UniqueName: \"kubernetes.io/projected/3c45448a-d2cb-4b9e-8588-9c6324577b8b-kube-api-access-9lbvq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:08.051246 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.051210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:08.101849 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.101270    2562 generic.go:356] "Generic (PLEG): container finished" podID="789e13b0-0f43-4814-8311-95da61ce53f1" containerID="de0aa2bea6d3b0badba704a2a3343779207f1ee70fa3249432d6c816f9858c52" exitCode=0
Jul 21 12:23:08.101849 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.101330    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh" event={"ID":"789e13b0-0f43-4814-8311-95da61ce53f1","Type":"ContainerDied","Data":"de0aa2bea6d3b0badba704a2a3343779207f1ee70fa3249432d6c816f9858c52"}
Jul 21 12:23:08.103464 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.103435    2562 generic.go:356] "Generic (PLEG): container finished" podID="56f76b3b-58fe-402b-9b02-938436473fc7" containerID="0922eff1941e7ab83d2527d8ecf2b715718719598c2560f5a4a29e21b5cc97a5" exitCode=0
Jul 21 12:23:08.103591 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.103558    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c" event={"ID":"56f76b3b-58fe-402b-9b02-938436473fc7","Type":"ContainerDied","Data":"0922eff1941e7ab83d2527d8ecf2b715718719598c2560f5a4a29e21b5cc97a5"}
Jul 21 12:23:08.103666 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.103599    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c" event={"ID":"56f76b3b-58fe-402b-9b02-938436473fc7","Type":"ContainerStarted","Data":"f6d3fc2047aba782c988032e1ed308afa58939759024b5371139c5489da49616"}
Jul 21 12:23:08.189653 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.189475    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"]
Jul 21 12:23:08.192114 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:23:08.192082    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c45448a_d2cb_4b9e_8588_9c6324577b8b.slice/crio-fe79bedfad6adffeae999f20c113f1073300354d01d566bf9d89f40e33e39eba WatchSource:0}: Error finding container fe79bedfad6adffeae999f20c113f1073300354d01d566bf9d89f40e33e39eba: Status 404 returned error can't find the container with id fe79bedfad6adffeae999f20c113f1073300354d01d566bf9d89f40e33e39eba
Jul 21 12:23:08.315702 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.315665    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"]
Jul 21 12:23:08.318990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.318972    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.328678 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.328646    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"]
Jul 21 12:23:08.408566 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.408529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.408740 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.408578    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.408740 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.408599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whbsw\" (UniqueName: \"kubernetes.io/projected/06de834d-245a-430b-b7cd-32d73861b933-kube-api-access-whbsw\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.509733 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.509647    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.509733 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.509700    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.509733 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.509718    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whbsw\" (UniqueName: \"kubernetes.io/projected/06de834d-245a-430b-b7cd-32d73861b933-kube-api-access-whbsw\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.510172 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.510150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.510217 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.510180    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.518579 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.518549    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whbsw\" (UniqueName: \"kubernetes.io/projected/06de834d-245a-430b-b7cd-32d73861b933-kube-api-access-whbsw\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.665100 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.665059    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:08.795625 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:08.795596    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"]
Jul 21 12:23:08.798130 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:23:08.798087    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06de834d_245a_430b_b7cd_32d73861b933.slice/crio-dcf2ab439f226cb6c4e46d74875e1aee6b8263e02a611d4418da4cfd78211b1c WatchSource:0}: Error finding container dcf2ab439f226cb6c4e46d74875e1aee6b8263e02a611d4418da4cfd78211b1c: Status 404 returned error can't find the container with id dcf2ab439f226cb6c4e46d74875e1aee6b8263e02a611d4418da4cfd78211b1c
Jul 21 12:23:09.110085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:09.110058    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c" event={"ID":"56f76b3b-58fe-402b-9b02-938436473fc7","Type":"ContainerStarted","Data":"f5e51e63b1d135379290b6d70a6716baf1d9e3496a06140a4e27590fe7a84a71"}
Jul 21 12:23:09.111642 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:09.111617    2562 generic.go:356] "Generic (PLEG): container finished" podID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerID="2882833df210df51e2677c115a23a17417590a90af06a45a662da9b9cda9d33d" exitCode=0
Jul 21 12:23:09.111749 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:09.111695    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh" event={"ID":"3c45448a-d2cb-4b9e-8588-9c6324577b8b","Type":"ContainerDied","Data":"2882833df210df51e2677c115a23a17417590a90af06a45a662da9b9cda9d33d"}
Jul 21 12:23:09.111749 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:09.111737    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh" event={"ID":"3c45448a-d2cb-4b9e-8588-9c6324577b8b","Type":"ContainerStarted","Data":"fe79bedfad6adffeae999f20c113f1073300354d01d566bf9d89f40e33e39eba"}
Jul 21 12:23:09.113679 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:09.113653    2562 generic.go:356] "Generic (PLEG): container finished" podID="06de834d-245a-430b-b7cd-32d73861b933" containerID="3a5732dbd53c786306426d6b56d05dd9c4c5e19f946daecc9eb838bf62249d8f" exitCode=0
Jul 21 12:23:09.113790 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:09.113686    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5" event={"ID":"06de834d-245a-430b-b7cd-32d73861b933","Type":"ContainerDied","Data":"3a5732dbd53c786306426d6b56d05dd9c4c5e19f946daecc9eb838bf62249d8f"}
Jul 21 12:23:09.113790 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:09.113730    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5" event={"ID":"06de834d-245a-430b-b7cd-32d73861b933","Type":"ContainerStarted","Data":"dcf2ab439f226cb6c4e46d74875e1aee6b8263e02a611d4418da4cfd78211b1c"}
Jul 21 12:23:10.119694 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:10.119666    2562 generic.go:356] "Generic (PLEG): container finished" podID="06de834d-245a-430b-b7cd-32d73861b933" containerID="644312e28a9dc76842948343b9609c12528e21ae79cf75bb73a1ec33026dcedc" exitCode=0
Jul 21 12:23:10.120041 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:10.119806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5" event={"ID":"06de834d-245a-430b-b7cd-32d73861b933","Type":"ContainerDied","Data":"644312e28a9dc76842948343b9609c12528e21ae79cf75bb73a1ec33026dcedc"}
Jul 21 12:23:10.121584 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:10.121560    2562 generic.go:356] "Generic (PLEG): container finished" podID="789e13b0-0f43-4814-8311-95da61ce53f1" containerID="b9f2bad607b1f95d1f104dd0eefbf5e054a7b648cfc44ab87dfe4faffc8acd24" exitCode=0
Jul 21 12:23:10.121677 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:10.121651    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh" event={"ID":"789e13b0-0f43-4814-8311-95da61ce53f1","Type":"ContainerDied","Data":"b9f2bad607b1f95d1f104dd0eefbf5e054a7b648cfc44ab87dfe4faffc8acd24"}
Jul 21 12:23:10.123487 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:10.123466    2562 generic.go:356] "Generic (PLEG): container finished" podID="56f76b3b-58fe-402b-9b02-938436473fc7" containerID="f5e51e63b1d135379290b6d70a6716baf1d9e3496a06140a4e27590fe7a84a71" exitCode=0
Jul 21 12:23:10.123591 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:10.123572    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c" event={"ID":"56f76b3b-58fe-402b-9b02-938436473fc7","Type":"ContainerDied","Data":"f5e51e63b1d135379290b6d70a6716baf1d9e3496a06140a4e27590fe7a84a71"}
Jul 21 12:23:10.125441 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:10.125420    2562 generic.go:356] "Generic (PLEG): container finished" podID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerID="3c8f0f1fdfc9f780113987962b3992ee4fb5165c9bf77dc3da4455aff2886bb8" exitCode=0
Jul 21 12:23:10.125500 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:10.125475    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh" event={"ID":"3c45448a-d2cb-4b9e-8588-9c6324577b8b","Type":"ContainerDied","Data":"3c8f0f1fdfc9f780113987962b3992ee4fb5165c9bf77dc3da4455aff2886bb8"}
Jul 21 12:23:11.131956 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:11.131921    2562 generic.go:356] "Generic (PLEG): container finished" podID="06de834d-245a-430b-b7cd-32d73861b933" containerID="178f7d2202c753185007c923a388444768e50ff0cf5006a95cc9e40ff1d42d11" exitCode=0
Jul 21 12:23:11.132375 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:11.132034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5" event={"ID":"06de834d-245a-430b-b7cd-32d73861b933","Type":"ContainerDied","Data":"178f7d2202c753185007c923a388444768e50ff0cf5006a95cc9e40ff1d42d11"}
Jul 21 12:23:11.134200 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:11.134175    2562 generic.go:356] "Generic (PLEG): container finished" podID="789e13b0-0f43-4814-8311-95da61ce53f1" containerID="9c8cb352b4f142a988f493f38f97f948f75ee24e0733b219fd16c63676c95aa0" exitCode=0
Jul 21 12:23:11.134321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:11.134251    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh" event={"ID":"789e13b0-0f43-4814-8311-95da61ce53f1","Type":"ContainerDied","Data":"9c8cb352b4f142a988f493f38f97f948f75ee24e0733b219fd16c63676c95aa0"}
Jul 21 12:23:11.136316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:11.136293    2562 generic.go:356] "Generic (PLEG): container finished" podID="56f76b3b-58fe-402b-9b02-938436473fc7" containerID="e42e52d90bdc39901fc26003e11c6208e151aa11fb0a8750c4f3fe5fea253376" exitCode=0
Jul 21 12:23:11.136421 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:11.136390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c" event={"ID":"56f76b3b-58fe-402b-9b02-938436473fc7","Type":"ContainerDied","Data":"e42e52d90bdc39901fc26003e11c6208e151aa11fb0a8750c4f3fe5fea253376"}
Jul 21 12:23:11.138263 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:11.138244    2562 generic.go:356] "Generic (PLEG): container finished" podID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerID="a3b36efe8020e954962135ee23f009a8178700a203fb602f22e03bab3e75e15c" exitCode=0
Jul 21 12:23:11.138336 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:11.138288    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh" event={"ID":"3c45448a-d2cb-4b9e-8588-9c6324577b8b","Type":"ContainerDied","Data":"a3b36efe8020e954962135ee23f009a8178700a203fb602f22e03bab3e75e15c"}
Jul 21 12:23:12.293445 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.293148    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:12.347438 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.347409    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:12.356953 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.356921    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:12.362872 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.362851    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:12.442306 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.442232    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-util\") pod \"56f76b3b-58fe-402b-9b02-938436473fc7\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") "
Jul 21 12:23:12.442306 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.442277    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-bundle\") pod \"56f76b3b-58fe-402b-9b02-938436473fc7\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") "
Jul 21 12:23:12.442483 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.442319    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-bundle\") pod \"789e13b0-0f43-4814-8311-95da61ce53f1\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") "
Jul 21 12:23:12.442483 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.442335    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-util\") pod \"789e13b0-0f43-4814-8311-95da61ce53f1\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") "
Jul 21 12:23:12.442483 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.442362    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52fv6\" (UniqueName: \"kubernetes.io/projected/56f76b3b-58fe-402b-9b02-938436473fc7-kube-api-access-52fv6\") pod \"56f76b3b-58fe-402b-9b02-938436473fc7\" (UID: \"56f76b3b-58fe-402b-9b02-938436473fc7\") "
Jul 21 12:23:12.442483 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.442390    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-util\") pod \"06de834d-245a-430b-b7cd-32d73861b933\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") "
Jul 21 12:23:12.442483 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.442460    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptw8l\" (UniqueName: \"kubernetes.io/projected/789e13b0-0f43-4814-8311-95da61ce53f1-kube-api-access-ptw8l\") pod \"789e13b0-0f43-4814-8311-95da61ce53f1\" (UID: \"789e13b0-0f43-4814-8311-95da61ce53f1\") "
Jul 21 12:23:12.442728 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.442497    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whbsw\" (UniqueName: \"kubernetes.io/projected/06de834d-245a-430b-b7cd-32d73861b933-kube-api-access-whbsw\") pod \"06de834d-245a-430b-b7cd-32d73861b933\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") "
Jul 21 12:23:12.442728 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.442558    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-bundle\") pod \"06de834d-245a-430b-b7cd-32d73861b933\" (UID: \"06de834d-245a-430b-b7cd-32d73861b933\") "
Jul 21 12:23:12.443335 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.443282    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-bundle" (OuterVolumeSpecName: "bundle") pod "06de834d-245a-430b-b7cd-32d73861b933" (UID: "06de834d-245a-430b-b7cd-32d73861b933"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:23:12.443431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.443345    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-bundle" (OuterVolumeSpecName: "bundle") pod "789e13b0-0f43-4814-8311-95da61ce53f1" (UID: "789e13b0-0f43-4814-8311-95da61ce53f1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:23:12.443431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.443309    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-bundle" (OuterVolumeSpecName: "bundle") pod "56f76b3b-58fe-402b-9b02-938436473fc7" (UID: "56f76b3b-58fe-402b-9b02-938436473fc7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:23:12.445229 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.445202    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06de834d-245a-430b-b7cd-32d73861b933-kube-api-access-whbsw" (OuterVolumeSpecName: "kube-api-access-whbsw") pod "06de834d-245a-430b-b7cd-32d73861b933" (UID: "06de834d-245a-430b-b7cd-32d73861b933"). InnerVolumeSpecName "kube-api-access-whbsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:23:12.445382 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.445202    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56f76b3b-58fe-402b-9b02-938436473fc7-kube-api-access-52fv6" (OuterVolumeSpecName: "kube-api-access-52fv6") pod "56f76b3b-58fe-402b-9b02-938436473fc7" (UID: "56f76b3b-58fe-402b-9b02-938436473fc7"). InnerVolumeSpecName "kube-api-access-52fv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:23:12.445533 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.445397    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/789e13b0-0f43-4814-8311-95da61ce53f1-kube-api-access-ptw8l" (OuterVolumeSpecName: "kube-api-access-ptw8l") pod "789e13b0-0f43-4814-8311-95da61ce53f1" (UID: "789e13b0-0f43-4814-8311-95da61ce53f1"). InnerVolumeSpecName "kube-api-access-ptw8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:23:12.450570 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.450538    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-util" (OuterVolumeSpecName: "util") pod "56f76b3b-58fe-402b-9b02-938436473fc7" (UID: "56f76b3b-58fe-402b-9b02-938436473fc7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:23:12.450570 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.450545    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-util" (OuterVolumeSpecName: "util") pod "06de834d-245a-430b-b7cd-32d73861b933" (UID: "06de834d-245a-430b-b7cd-32d73861b933"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:23:12.473399 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.473375    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-util" (OuterVolumeSpecName: "util") pod "789e13b0-0f43-4814-8311-95da61ce53f1" (UID: "789e13b0-0f43-4814-8311-95da61ce53f1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:23:12.543247 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543218    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lbvq\" (UniqueName: \"kubernetes.io/projected/3c45448a-d2cb-4b9e-8588-9c6324577b8b-kube-api-access-9lbvq\") pod \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") "
Jul 21 12:23:12.543400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543271    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-util\") pod \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") "
Jul 21 12:23:12.543400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543344    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-bundle\") pod \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\" (UID: \"3c45448a-d2cb-4b9e-8588-9c6324577b8b\") "
Jul 21 12:23:12.543547 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543530    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whbsw\" (UniqueName: \"kubernetes.io/projected/06de834d-245a-430b-b7cd-32d73861b933-kube-api-access-whbsw\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.543629 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543551    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.543629 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543564    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-util\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.543629 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543576    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f76b3b-58fe-402b-9b02-938436473fc7-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.543629 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543587    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.543629 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543597    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/789e13b0-0f43-4814-8311-95da61ce53f1-util\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.543629 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543610    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52fv6\" (UniqueName: \"kubernetes.io/projected/56f76b3b-58fe-402b-9b02-938436473fc7-kube-api-access-52fv6\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.543629 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543623    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06de834d-245a-430b-b7cd-32d73861b933-util\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.543899 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543636    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptw8l\" (UniqueName: \"kubernetes.io/projected/789e13b0-0f43-4814-8311-95da61ce53f1-kube-api-access-ptw8l\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.543993 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.543966    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-bundle" (OuterVolumeSpecName: "bundle") pod "3c45448a-d2cb-4b9e-8588-9c6324577b8b" (UID: "3c45448a-d2cb-4b9e-8588-9c6324577b8b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:23:12.546276 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.546250    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c45448a-d2cb-4b9e-8588-9c6324577b8b-kube-api-access-9lbvq" (OuterVolumeSpecName: "kube-api-access-9lbvq") pod "3c45448a-d2cb-4b9e-8588-9c6324577b8b" (UID: "3c45448a-d2cb-4b9e-8588-9c6324577b8b"). InnerVolumeSpecName "kube-api-access-9lbvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:23:12.548877 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.548854    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-util" (OuterVolumeSpecName: "util") pod "3c45448a-d2cb-4b9e-8588-9c6324577b8b" (UID: "3c45448a-d2cb-4b9e-8588-9c6324577b8b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:23:12.644522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.644483    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lbvq\" (UniqueName: \"kubernetes.io/projected/3c45448a-d2cb-4b9e-8588-9c6324577b8b-kube-api-access-9lbvq\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.644522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.644517    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-util\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:12.644522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:12.644528    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c45448a-d2cb-4b9e-8588-9c6324577b8b-bundle\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:13.149653 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.149621    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c"
Jul 21 12:23:13.149653 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.149647    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c" event={"ID":"56f76b3b-58fe-402b-9b02-938436473fc7","Type":"ContainerDied","Data":"f6d3fc2047aba782c988032e1ed308afa58939759024b5371139c5489da49616"}
Jul 21 12:23:13.149901 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.149734    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6d3fc2047aba782c988032e1ed308afa58939759024b5371139c5489da49616"
Jul 21 12:23:13.151548 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.151525    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh"
Jul 21 12:23:13.151667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.151555    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh" event={"ID":"3c45448a-d2cb-4b9e-8588-9c6324577b8b","Type":"ContainerDied","Data":"fe79bedfad6adffeae999f20c113f1073300354d01d566bf9d89f40e33e39eba"}
Jul 21 12:23:13.151667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.151596    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe79bedfad6adffeae999f20c113f1073300354d01d566bf9d89f40e33e39eba"
Jul 21 12:23:13.153396 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.153376    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5"
Jul 21 12:23:13.153485 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.153415    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5" event={"ID":"06de834d-245a-430b-b7cd-32d73861b933","Type":"ContainerDied","Data":"dcf2ab439f226cb6c4e46d74875e1aee6b8263e02a611d4418da4cfd78211b1c"}
Jul 21 12:23:13.153485 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.153456    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcf2ab439f226cb6c4e46d74875e1aee6b8263e02a611d4418da4cfd78211b1c"
Jul 21 12:23:13.155327 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.155307    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh" event={"ID":"789e13b0-0f43-4814-8311-95da61ce53f1","Type":"ContainerDied","Data":"cfd399fea46090800eedc0a1c49aa79c77d81a99d21aa41a5e994de5171d4c4f"}
Jul 21 12:23:13.155398 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.155339    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh"
Jul 21 12:23:13.155398 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:13.155347    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfd399fea46090800eedc0a1c49aa79c77d81a99d21aa41a5e994de5171d4c4f"
Jul 21 12:23:44.089667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.089629    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"]
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090033    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="789e13b0-0f43-4814-8311-95da61ce53f1" containerName="extract"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090054    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="789e13b0-0f43-4814-8311-95da61ce53f1" containerName="extract"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090074    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f76b3b-58fe-402b-9b02-938436473fc7" containerName="pull"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090081    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f76b3b-58fe-402b-9b02-938436473fc7" containerName="pull"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090089    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="789e13b0-0f43-4814-8311-95da61ce53f1" containerName="pull"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090097    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="789e13b0-0f43-4814-8311-95da61ce53f1" containerName="pull"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090107    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerName="pull"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090114    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerName="pull"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090124    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="789e13b0-0f43-4814-8311-95da61ce53f1" containerName="util"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090131    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="789e13b0-0f43-4814-8311-95da61ce53f1" containerName="util"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090141    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f76b3b-58fe-402b-9b02-938436473fc7" containerName="extract"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090149    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f76b3b-58fe-402b-9b02-938436473fc7" containerName="extract"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090164    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f76b3b-58fe-402b-9b02-938436473fc7" containerName="util"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090172    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f76b3b-58fe-402b-9b02-938436473fc7" containerName="util"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090180    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06de834d-245a-430b-b7cd-32d73861b933" containerName="extract"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090188    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="06de834d-245a-430b-b7cd-32d73861b933" containerName="extract"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090200    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06de834d-245a-430b-b7cd-32d73861b933" containerName="pull"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090208    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="06de834d-245a-430b-b7cd-32d73861b933" containerName="pull"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090222    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerName="util"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090230    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerName="util"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090237    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06de834d-245a-430b-b7cd-32d73861b933" containerName="util"
Jul 21 12:23:44.090238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090245    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="06de834d-245a-430b-b7cd-32d73861b933" containerName="util"
Jul 21 12:23:44.091264 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090256    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerName="extract"
Jul 21 12:23:44.091264 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090264    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerName="extract"
Jul 21 12:23:44.091264 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090341    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="06de834d-245a-430b-b7cd-32d73861b933" containerName="extract"
Jul 21 12:23:44.091264 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090352    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c45448a-d2cb-4b9e-8588-9c6324577b8b" containerName="extract"
Jul 21 12:23:44.091264 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090363    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="789e13b0-0f43-4814-8311-95da61ce53f1" containerName="extract"
Jul 21 12:23:44.091264 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.090373    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="56f76b3b-58fe-402b-9b02-938436473fc7" containerName="extract"
Jul 21 12:23:44.094975 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.094955    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.099763 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.099740    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 12:23:44.099877 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.099792    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 12:23:44.100326 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.100309    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ns8wt\""
Jul 21 12:23:44.108039 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.108017    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"]
Jul 21 12:23:44.184382 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.184351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbxj8\" (UniqueName: \"kubernetes.io/projected/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-kube-api-access-kbxj8\") pod \"kuadrant-operator-controller-manager-55467d9f7-cdjtg\" (UID: \"ae8eff7f-f043-4084-bdb8-4f5bbe35d748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.184518 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.184398    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-cdjtg\" (UID: \"ae8eff7f-f043-4084-bdb8-4f5bbe35d748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.191371 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.191343    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"]
Jul 21 12:23:44.191536 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:23:44.191519    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-kbxj8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg" podUID="ae8eff7f-f043-4084-bdb8-4f5bbe35d748"
Jul 21 12:23:44.220829 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.220802    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"]
Jul 21 12:23:44.229617 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.229593    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"]
Jul 21 12:23:44.285258 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.285224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbxj8\" (UniqueName: \"kubernetes.io/projected/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-kube-api-access-kbxj8\") pod \"kuadrant-operator-controller-manager-55467d9f7-cdjtg\" (UID: \"ae8eff7f-f043-4084-bdb8-4f5bbe35d748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.285423 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.285272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-cdjtg\" (UID: \"ae8eff7f-f043-4084-bdb8-4f5bbe35d748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.285660 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.285640    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-cdjtg\" (UID: \"ae8eff7f-f043-4084-bdb8-4f5bbe35d748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.287331 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:23:44.287314    2562 projected.go:196] Error preparing data for projected volume kube-api-access-kbxj8 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-138-237.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object
Jul 21 12:23:44.287423 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:23:44.287385    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-kube-api-access-kbxj8 podName:ae8eff7f-f043-4084-bdb8-4f5bbe35d748 nodeName:}" failed. No retries permitted until 2026-07-21 12:23:44.787364637 +0000 UTC m=+748.845795635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kbxj8" (UniqueName: "kubernetes.io/projected/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-kube-api-access-kbxj8") pod "kuadrant-operator-controller-manager-55467d9f7-cdjtg" (UID: "ae8eff7f-f043-4084-bdb8-4f5bbe35d748") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-138-237.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object
Jul 21 12:23:44.307698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.307667    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.309998 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:23:44.309968    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cdjtg\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="ae8eff7f-f043-4084-bdb8-4f5bbe35d748" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.314531 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.314511    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.316598 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:23:44.316572    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cdjtg\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="ae8eff7f-f043-4084-bdb8-4f5bbe35d748" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:44.386117 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.386048    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-extensions-socket-volume\") pod \"ae8eff7f-f043-4084-bdb8-4f5bbe35d748\" (UID: \"ae8eff7f-f043-4084-bdb8-4f5bbe35d748\") "
Jul 21 12:23:44.386237 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.386213    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbxj8\" (UniqueName: \"kubernetes.io/projected/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-kube-api-access-kbxj8\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:44.386315 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.386299    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ae8eff7f-f043-4084-bdb8-4f5bbe35d748" (UID: "ae8eff7f-f043-4084-bdb8-4f5bbe35d748"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:23:44.416037 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.416013    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae8eff7f-f043-4084-bdb8-4f5bbe35d748" path="/var/lib/kubelet/pods/ae8eff7f-f043-4084-bdb8-4f5bbe35d748/volumes"
Jul 21 12:23:44.487076 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:44.487048    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae8eff7f-f043-4084-bdb8-4f5bbe35d748-extensions-socket-volume\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:23:45.312523 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:23:45.312489    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:45.318368 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:23:45.318340    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cdjtg\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="ae8eff7f-f043-4084-bdb8-4f5bbe35d748" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:23:46.416226 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:23:46.416187    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cdjtg\" is forbidden: User \"system:node:ip-10-0-138-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-237.ec2.internal' and this object" podUID="ae8eff7f-f043-4084-bdb8-4f5bbe35d748" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cdjtg"
Jul 21 12:24:33.010701 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.010669    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:24:33.013905 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.013875    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:24:33.016427 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.016392    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 12:24:33.016552 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.016392    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 21 12:24:33.016552 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.016392    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mslb2\""
Jul 21 12:24:33.017224 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.017209    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 12:24:33.022877 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.022853    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:24:33.077095 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.077061    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:24:33.169229 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.169192    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzgdr\" (UniqueName: \"kubernetes.io/projected/f3f4473a-baac-414d-9ccc-f561b6286305-kube-api-access-zzgdr\") pod \"limitador-limitador-54d6d988d5-wjrqj\" (UID: \"f3f4473a-baac-414d-9ccc-f561b6286305\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:24:33.169405 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.169248    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3f4473a-baac-414d-9ccc-f561b6286305-config-file\") pod \"limitador-limitador-54d6d988d5-wjrqj\" (UID: \"f3f4473a-baac-414d-9ccc-f561b6286305\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:24:33.270251 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.270163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3f4473a-baac-414d-9ccc-f561b6286305-config-file\") pod \"limitador-limitador-54d6d988d5-wjrqj\" (UID: \"f3f4473a-baac-414d-9ccc-f561b6286305\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:24:33.270404 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.270256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzgdr\" (UniqueName: \"kubernetes.io/projected/f3f4473a-baac-414d-9ccc-f561b6286305-kube-api-access-zzgdr\") pod \"limitador-limitador-54d6d988d5-wjrqj\" (UID: \"f3f4473a-baac-414d-9ccc-f561b6286305\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:24:33.270798 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.270753    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3f4473a-baac-414d-9ccc-f561b6286305-config-file\") pod \"limitador-limitador-54d6d988d5-wjrqj\" (UID: \"f3f4473a-baac-414d-9ccc-f561b6286305\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:24:33.278864 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.278842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzgdr\" (UniqueName: \"kubernetes.io/projected/f3f4473a-baac-414d-9ccc-f561b6286305-kube-api-access-zzgdr\") pod \"limitador-limitador-54d6d988d5-wjrqj\" (UID: \"f3f4473a-baac-414d-9ccc-f561b6286305\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:24:33.325000 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.324956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:24:33.456709 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.456683    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:24:33.458808 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:24:33.458759    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f4473a_baac_414d_9ccc_f561b6286305.slice/crio-8d747e13d8d42132b6219f628ce0781850cd6b9364589412bc95ecc9aa5f5904 WatchSource:0}: Error finding container 8d747e13d8d42132b6219f628ce0781850cd6b9364589412bc95ecc9aa5f5904: Status 404 returned error can't find the container with id 8d747e13d8d42132b6219f628ce0781850cd6b9364589412bc95ecc9aa5f5904
Jul 21 12:24:33.526804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:33.526722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj" event={"ID":"f3f4473a-baac-414d-9ccc-f561b6286305","Type":"ContainerStarted","Data":"8d747e13d8d42132b6219f628ce0781850cd6b9364589412bc95ecc9aa5f5904"}
Jul 21 12:24:36.542139 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:36.542045    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj" event={"ID":"f3f4473a-baac-414d-9ccc-f561b6286305","Type":"ContainerStarted","Data":"1ac99abd41c62b394b20231f4dcd29f57ced32883f95d76ac074b15acec435f6"}
Jul 21 12:24:36.542550 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:36.542158    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:24:36.559893 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:36.559853    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj" podStartSLOduration=1.923487777 podStartE2EDuration="4.559839952s" totalImagesPullingTime="2.636352175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:24:36.558403224 +0000 UTC m=+800.616834228" watchObservedRunningTime="2026-07-21 12:24:36.559839952 +0000 UTC m=+800.618270956"
Jul 21 12:24:47.547380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:24:47.547339    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"
Jul 21 12:25:07.846718 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:07.846685    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c646458b4-zc5tc"]
Jul 21 12:25:07.849944 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:07.849926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:07.852584 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:07.852560    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 12:25:07.852725 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:07.852560    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hc8m7\""
Jul 21 12:25:07.852725 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:07.852564    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 12:25:07.859948 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:07.859922    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c646458b4-zc5tc"]
Jul 21 12:25:07.938875 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:07.938832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64f1f99f-f452-44c1-9d45-24ef6f8e392c-webhook-cert\") pod \"maas-controller-6c646458b4-zc5tc\" (UID: \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\") " pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:07.939053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:07.938892    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfkmg\" (UniqueName: \"kubernetes.io/projected/64f1f99f-f452-44c1-9d45-24ef6f8e392c-kube-api-access-bfkmg\") pod \"maas-controller-6c646458b4-zc5tc\" (UID: \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\") " pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:08.040039 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:08.040000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64f1f99f-f452-44c1-9d45-24ef6f8e392c-webhook-cert\") pod \"maas-controller-6c646458b4-zc5tc\" (UID: \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\") " pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:08.040198 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:08.040081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfkmg\" (UniqueName: \"kubernetes.io/projected/64f1f99f-f452-44c1-9d45-24ef6f8e392c-kube-api-access-bfkmg\") pod \"maas-controller-6c646458b4-zc5tc\" (UID: \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\") " pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:08.042724 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:08.042701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64f1f99f-f452-44c1-9d45-24ef6f8e392c-webhook-cert\") pod \"maas-controller-6c646458b4-zc5tc\" (UID: \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\") " pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:08.048235 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:08.048210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfkmg\" (UniqueName: \"kubernetes.io/projected/64f1f99f-f452-44c1-9d45-24ef6f8e392c-kube-api-access-bfkmg\") pod \"maas-controller-6c646458b4-zc5tc\" (UID: \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\") " pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:08.161943 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:08.161854    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:08.293928 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:08.293902    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c646458b4-zc5tc"]
Jul 21 12:25:08.296377 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:25:08.296333    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f1f99f_f452_44c1_9d45_24ef6f8e392c.slice/crio-0f5a3bf30a4c65f92a1cfc9423e61ef93b5e36b4cd499301701016b8aecca8ac WatchSource:0}: Error finding container 0f5a3bf30a4c65f92a1cfc9423e61ef93b5e36b4cd499301701016b8aecca8ac: Status 404 returned error can't find the container with id 0f5a3bf30a4c65f92a1cfc9423e61ef93b5e36b4cd499301701016b8aecca8ac
Jul 21 12:25:08.689424 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:08.689393    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c646458b4-zc5tc" event={"ID":"64f1f99f-f452-44c1-9d45-24ef6f8e392c","Type":"ContainerStarted","Data":"0f5a3bf30a4c65f92a1cfc9423e61ef93b5e36b4cd499301701016b8aecca8ac"}
Jul 21 12:25:11.706485 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:11.706446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c646458b4-zc5tc" event={"ID":"64f1f99f-f452-44c1-9d45-24ef6f8e392c","Type":"ContainerStarted","Data":"19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d"}
Jul 21 12:25:11.706887 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:11.706509    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:11.724691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:11.724646    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c646458b4-zc5tc" podStartSLOduration=2.188098545 podStartE2EDuration="4.724631684s" totalImagesPullingTime="2.536533139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:25:11.723882706 +0000 UTC m=+835.782313711" watchObservedRunningTime="2026-07-21 12:25:11.724631684 +0000 UTC m=+835.783062739"
Jul 21 12:25:22.222649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.222615    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c646458b4-zc5tc"]
Jul 21 12:25:22.223058 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.222869    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c646458b4-zc5tc" podUID="64f1f99f-f452-44c1-9d45-24ef6f8e392c" containerName="manager" containerID="cri-o://19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d" gracePeriod=30
Jul 21 12:25:22.224631 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.224611    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:22.470462 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.470431    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:22.567226 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.567185    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64f1f99f-f452-44c1-9d45-24ef6f8e392c-webhook-cert\") pod \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\" (UID: \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\") "
Jul 21 12:25:22.567389 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.567259    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfkmg\" (UniqueName: \"kubernetes.io/projected/64f1f99f-f452-44c1-9d45-24ef6f8e392c-kube-api-access-bfkmg\") pod \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\" (UID: \"64f1f99f-f452-44c1-9d45-24ef6f8e392c\") "
Jul 21 12:25:22.569576 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.569535    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64f1f99f-f452-44c1-9d45-24ef6f8e392c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "64f1f99f-f452-44c1-9d45-24ef6f8e392c" (UID: "64f1f99f-f452-44c1-9d45-24ef6f8e392c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:25:22.569576 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.569559    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64f1f99f-f452-44c1-9d45-24ef6f8e392c-kube-api-access-bfkmg" (OuterVolumeSpecName: "kube-api-access-bfkmg") pod "64f1f99f-f452-44c1-9d45-24ef6f8e392c" (UID: "64f1f99f-f452-44c1-9d45-24ef6f8e392c"). InnerVolumeSpecName "kube-api-access-bfkmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:25:22.668251 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.668216    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64f1f99f-f452-44c1-9d45-24ef6f8e392c-webhook-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:25:22.668251 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.668244    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfkmg\" (UniqueName: \"kubernetes.io/projected/64f1f99f-f452-44c1-9d45-24ef6f8e392c-kube-api-access-bfkmg\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:25:22.755911 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.755875    2562 generic.go:356] "Generic (PLEG): container finished" podID="64f1f99f-f452-44c1-9d45-24ef6f8e392c" containerID="19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d" exitCode=0
Jul 21 12:25:22.756068 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.755922    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c646458b4-zc5tc"
Jul 21 12:25:22.756068 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.755960    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c646458b4-zc5tc" event={"ID":"64f1f99f-f452-44c1-9d45-24ef6f8e392c","Type":"ContainerDied","Data":"19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d"}
Jul 21 12:25:22.756068 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.756015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c646458b4-zc5tc" event={"ID":"64f1f99f-f452-44c1-9d45-24ef6f8e392c","Type":"ContainerDied","Data":"0f5a3bf30a4c65f92a1cfc9423e61ef93b5e36b4cd499301701016b8aecca8ac"}
Jul 21 12:25:22.756068 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.756038    2562 scope.go:117] "RemoveContainer" containerID="19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d"
Jul 21 12:25:22.764661 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.764641    2562 scope.go:117] "RemoveContainer" containerID="19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d"
Jul 21 12:25:22.764936 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:25:22.764917    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d\": container with ID starting with 19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d not found: ID does not exist" containerID="19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d"
Jul 21 12:25:22.764990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.764945    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d"} err="failed to get container status \"19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d\": rpc error: code = NotFound desc = could not find container \"19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d\": container with ID starting with 19d0818e34d4c63091ec9bedc2d4fd0aee935186bcdec8f80ea81cdd39316d7d not found: ID does not exist"
Jul 21 12:25:22.776336 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.776304    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c646458b4-zc5tc"]
Jul 21 12:25:22.781287 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:22.781267    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c646458b4-zc5tc"]
Jul 21 12:25:24.416140 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:24.416109    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64f1f99f-f452-44c1-9d45-24ef6f8e392c" path="/var/lib/kubelet/pods/64f1f99f-f452-44c1-9d45-24ef6f8e392c/volumes"
Jul 21 12:25:43.765394 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.765359    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-hrhxm"]
Jul 21 12:25:43.765834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.765632    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64f1f99f-f452-44c1-9d45-24ef6f8e392c" containerName="manager"
Jul 21 12:25:43.765834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.765643    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="64f1f99f-f452-44c1-9d45-24ef6f8e392c" containerName="manager"
Jul 21 12:25:43.765834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.765703    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="64f1f99f-f452-44c1-9d45-24ef6f8e392c" containerName="manager"
Jul 21 12:25:43.771996 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.771977    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:25:43.774460 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.774441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 21 12:25:43.774534 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.774442    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nx4lk\""
Jul 21 12:25:43.780283 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.780255    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-hrhxm"]
Jul 21 12:25:43.824256 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.824223    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:25:43.837691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.837664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ggjq\" (UniqueName: \"kubernetes.io/projected/b378e830-e542-4069-8f28-ff2ab7443874-kube-api-access-8ggjq\") pod \"payload-processing-5bf47cd988-hrhxm\" (UID: \"b378e830-e542-4069-8f28-ff2ab7443874\") " pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:25:43.837842 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.837812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b378e830-e542-4069-8f28-ff2ab7443874-plugins-config-volume\") pod \"payload-processing-5bf47cd988-hrhxm\" (UID: \"b378e830-e542-4069-8f28-ff2ab7443874\") " pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:25:43.938300 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.938264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b378e830-e542-4069-8f28-ff2ab7443874-plugins-config-volume\") pod \"payload-processing-5bf47cd988-hrhxm\" (UID: \"b378e830-e542-4069-8f28-ff2ab7443874\") " pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:25:43.938461 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.938313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ggjq\" (UniqueName: \"kubernetes.io/projected/b378e830-e542-4069-8f28-ff2ab7443874-kube-api-access-8ggjq\") pod \"payload-processing-5bf47cd988-hrhxm\" (UID: \"b378e830-e542-4069-8f28-ff2ab7443874\") " pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:25:43.939006 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.938984    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b378e830-e542-4069-8f28-ff2ab7443874-plugins-config-volume\") pod \"payload-processing-5bf47cd988-hrhxm\" (UID: \"b378e830-e542-4069-8f28-ff2ab7443874\") " pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:25:43.948841 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:43.948811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ggjq\" (UniqueName: \"kubernetes.io/projected/b378e830-e542-4069-8f28-ff2ab7443874-kube-api-access-8ggjq\") pod \"payload-processing-5bf47cd988-hrhxm\" (UID: \"b378e830-e542-4069-8f28-ff2ab7443874\") " pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:25:44.082905 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:44.082870    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:25:44.214404 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:44.214379    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-hrhxm"]
Jul 21 12:25:44.216545 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:25:44.216519    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb378e830_e542_4069_8f28_ff2ab7443874.slice/crio-8d572f6ea571eda90e6773fa1cd6855d33e58283bc8ecf8932897ebcc540a610 WatchSource:0}: Error finding container 8d572f6ea571eda90e6773fa1cd6855d33e58283bc8ecf8932897ebcc540a610: Status 404 returned error can't find the container with id 8d572f6ea571eda90e6773fa1cd6855d33e58283bc8ecf8932897ebcc540a610
Jul 21 12:25:44.218400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:44.218382    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:25:44.863680 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:44.863636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm" event={"ID":"b378e830-e542-4069-8f28-ff2ab7443874","Type":"ContainerStarted","Data":"8d572f6ea571eda90e6773fa1cd6855d33e58283bc8ecf8932897ebcc540a610"}
Jul 21 12:25:45.869438 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:45.869403    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm" event={"ID":"b378e830-e542-4069-8f28-ff2ab7443874","Type":"ContainerStarted","Data":"73a5ac7db85fae8e9377ecde0545544665cb2684d0bba2263b3cd46cf33bfe03"}
Jul 21 12:25:45.869865 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:45.869494    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:25:45.884976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:45.884918    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm" podStartSLOduration=1.333055014 podStartE2EDuration="2.884898361s" totalImagesPullingTime="1.551843347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:25:45.883860489 +0000 UTC m=+869.942291494" watchObservedRunningTime="2026-07-21 12:25:45.884898361 +0000 UTC m=+869.943329374"
Jul 21 12:25:56.877183 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:25:56.877152    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-hrhxm"
Jul 21 12:26:08.685978 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:08.685893    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:26:12.989922 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:12.989883    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:26:34.097011 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:34.096968    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:26:49.896882 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:49.896839    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:26:52.489376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.489344    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"]
Jul 21 12:26:52.492428 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.492412    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.494811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.494789    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 12:26:52.495655 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.495637    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 12:26:52.495761 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.495645    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 21 12:26:52.495761 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.495717    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-46ngs\""
Jul 21 12:26:52.503247 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.503226    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"]
Jul 21 12:26:52.602925 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.602889    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.603106 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.602933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wdsv\" (UniqueName: \"kubernetes.io/projected/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-kube-api-access-4wdsv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.603106 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.603004    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.603106 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.603074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.603208 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.603109    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.603208 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.603133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.704167 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.704132    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.704167 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.704168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.704428 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.704203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.704428 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.704223    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wdsv\" (UniqueName: \"kubernetes.io/projected/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-kube-api-access-4wdsv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.704428 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.704254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.704428 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.704282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.704628 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.704599    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.704680 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.704627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.704680 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.704670    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.706643 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.706620    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.706925 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.706906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.713348 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.713322    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wdsv\" (UniqueName: \"kubernetes.io/projected/5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8-kube-api-access-4wdsv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qbhs4\" (UID: \"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.803123 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.803029    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:26:52.938992 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:52.938965    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"]
Jul 21 12:26:52.941608 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:26:52.941580    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf0343f_6b3f_4d6e_b904_5dcaf13ef1b8.slice/crio-03d9e70e00446bfcc1b71aeca4b0a0561d597f7ff82cfc931d04b76153457619 WatchSource:0}: Error finding container 03d9e70e00446bfcc1b71aeca4b0a0561d597f7ff82cfc931d04b76153457619: Status 404 returned error can't find the container with id 03d9e70e00446bfcc1b71aeca4b0a0561d597f7ff82cfc931d04b76153457619
Jul 21 12:26:53.182878 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:53.182832    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4" event={"ID":"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8","Type":"ContainerStarted","Data":"03d9e70e00446bfcc1b71aeca4b0a0561d597f7ff82cfc931d04b76153457619"}
Jul 21 12:26:54.189970 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:54.189930    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4" event={"ID":"5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8","Type":"ContainerStarted","Data":"f86b11d73666edff5614ef8193ce49ec97aa45f1ff936d853bbfa6dd44168889"}
Jul 21 12:26:54.209681 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:54.209592    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4" podStartSLOduration=1.200198097 podStartE2EDuration="2.209577263s" totalImagesPullingTime="1.009379166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:26:54.207073569 +0000 UTC m=+938.265504573" watchObservedRunningTime="2026-07-21 12:26:54.209577263 +0000 UTC m=+938.268008261"
Jul 21 12:26:54.993968 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:26:54.993929    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:27:02.634379 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:27:02.634342    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:27:02.803743 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:27:02.803707    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:27:02.803931 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:27:02.803863    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:27:02.821163 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:27:02.821133    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:27:03.244017 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:27:03.243981    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qbhs4"
Jul 21 12:28:18.599000 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:28:18.598962    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:28:30.001665 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:28:30.001625    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:28:38.194643 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:28:38.194608    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:28:48.723626 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:28:48.723588    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:29:42.214283 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.214250    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7966fbf488-mb5tq"]
Jul 21 12:29:42.217381 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.217362    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:42.220626 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.220597    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 12:29:42.220748 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.220598    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 12:29:42.220748 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.220606    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hc8m7\""
Jul 21 12:29:42.226355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.226310    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7966fbf488-mb5tq"]
Jul 21 12:29:42.304129 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.304097    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bbb11a5-38db-4812-93c7-76ce24416fb1-webhook-cert\") pod \"maas-controller-7966fbf488-mb5tq\" (UID: \"7bbb11a5-38db-4812-93c7-76ce24416fb1\") " pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:42.304220 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.304210    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfqk6\" (UniqueName: \"kubernetes.io/projected/7bbb11a5-38db-4812-93c7-76ce24416fb1-kube-api-access-jfqk6\") pod \"maas-controller-7966fbf488-mb5tq\" (UID: \"7bbb11a5-38db-4812-93c7-76ce24416fb1\") " pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:42.404953 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.404908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfqk6\" (UniqueName: \"kubernetes.io/projected/7bbb11a5-38db-4812-93c7-76ce24416fb1-kube-api-access-jfqk6\") pod \"maas-controller-7966fbf488-mb5tq\" (UID: \"7bbb11a5-38db-4812-93c7-76ce24416fb1\") " pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:42.405118 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.404967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bbb11a5-38db-4812-93c7-76ce24416fb1-webhook-cert\") pod \"maas-controller-7966fbf488-mb5tq\" (UID: \"7bbb11a5-38db-4812-93c7-76ce24416fb1\") " pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:42.407628 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.407608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bbb11a5-38db-4812-93c7-76ce24416fb1-webhook-cert\") pod \"maas-controller-7966fbf488-mb5tq\" (UID: \"7bbb11a5-38db-4812-93c7-76ce24416fb1\") " pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:42.414597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.414569    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfqk6\" (UniqueName: \"kubernetes.io/projected/7bbb11a5-38db-4812-93c7-76ce24416fb1-kube-api-access-jfqk6\") pod \"maas-controller-7966fbf488-mb5tq\" (UID: \"7bbb11a5-38db-4812-93c7-76ce24416fb1\") " pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:42.529156 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.529072    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:42.668938 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.668911    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7966fbf488-mb5tq"]
Jul 21 12:29:42.670938 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:29:42.670905    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bbb11a5_38db_4812_93c7_76ce24416fb1.slice/crio-e2bfe2517b44ead5dac3ff9036828c7f7a233f833cc35e47edbaeee514b269a9 WatchSource:0}: Error finding container e2bfe2517b44ead5dac3ff9036828c7f7a233f833cc35e47edbaeee514b269a9: Status 404 returned error can't find the container with id e2bfe2517b44ead5dac3ff9036828c7f7a233f833cc35e47edbaeee514b269a9
Jul 21 12:29:42.989029 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:42.988987    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-mb5tq" event={"ID":"7bbb11a5-38db-4812-93c7-76ce24416fb1","Type":"ContainerStarted","Data":"e2bfe2517b44ead5dac3ff9036828c7f7a233f833cc35e47edbaeee514b269a9"}
Jul 21 12:29:43.994821 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:43.994767    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-mb5tq" event={"ID":"7bbb11a5-38db-4812-93c7-76ce24416fb1","Type":"ContainerStarted","Data":"c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781"}
Jul 21 12:29:43.995188 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:43.994905    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:44.014079 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:44.014029    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7966fbf488-mb5tq" podStartSLOduration=1.576244447 podStartE2EDuration="2.01401018s" totalImagesPullingTime="437.765733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:44.012217966 +0000 UTC m=+1108.070648981" watchObservedRunningTime="2026-07-21 12:29:44.01401018 +0000 UTC m=+1108.072441187"
Jul 21 12:29:55.002372 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:55.002338    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:59.618493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:59.618454    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7966fbf488-mb5tq"]
Jul 21 12:29:59.618914 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:59.618638    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7966fbf488-mb5tq" podUID="7bbb11a5-38db-4812-93c7-76ce24416fb1" containerName="manager" containerID="cri-o://c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781" gracePeriod=30
Jul 21 12:29:59.860049 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:59.860022    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:29:59.956296 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:59.956209    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfqk6\" (UniqueName: \"kubernetes.io/projected/7bbb11a5-38db-4812-93c7-76ce24416fb1-kube-api-access-jfqk6\") pod \"7bbb11a5-38db-4812-93c7-76ce24416fb1\" (UID: \"7bbb11a5-38db-4812-93c7-76ce24416fb1\") "
Jul 21 12:29:59.956296 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:59.956261    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bbb11a5-38db-4812-93c7-76ce24416fb1-webhook-cert\") pod \"7bbb11a5-38db-4812-93c7-76ce24416fb1\" (UID: \"7bbb11a5-38db-4812-93c7-76ce24416fb1\") "
Jul 21 12:29:59.958475 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:59.958445    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bbb11a5-38db-4812-93c7-76ce24416fb1-kube-api-access-jfqk6" (OuterVolumeSpecName: "kube-api-access-jfqk6") pod "7bbb11a5-38db-4812-93c7-76ce24416fb1" (UID: "7bbb11a5-38db-4812-93c7-76ce24416fb1"). InnerVolumeSpecName "kube-api-access-jfqk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:29:59.958475 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:29:59.958450    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bbb11a5-38db-4812-93c7-76ce24416fb1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7bbb11a5-38db-4812-93c7-76ce24416fb1" (UID: "7bbb11a5-38db-4812-93c7-76ce24416fb1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:30:00.057645 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.057603    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfqk6\" (UniqueName: \"kubernetes.io/projected/7bbb11a5-38db-4812-93c7-76ce24416fb1-kube-api-access-jfqk6\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:30:00.057645 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.057638    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7bbb11a5-38db-4812-93c7-76ce24416fb1-webhook-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:30:00.070830 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.070800    2562 generic.go:356] "Generic (PLEG): container finished" podID="7bbb11a5-38db-4812-93c7-76ce24416fb1" containerID="c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781" exitCode=0
Jul 21 12:30:00.070830 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.070834    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-mb5tq"
Jul 21 12:30:00.071015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.070871    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-mb5tq" event={"ID":"7bbb11a5-38db-4812-93c7-76ce24416fb1","Type":"ContainerDied","Data":"c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781"}
Jul 21 12:30:00.071015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.070926    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-mb5tq" event={"ID":"7bbb11a5-38db-4812-93c7-76ce24416fb1","Type":"ContainerDied","Data":"e2bfe2517b44ead5dac3ff9036828c7f7a233f833cc35e47edbaeee514b269a9"}
Jul 21 12:30:00.071015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.070949    2562 scope.go:117] "RemoveContainer" containerID="c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781"
Jul 21 12:30:00.080090 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.080073    2562 scope.go:117] "RemoveContainer" containerID="c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781"
Jul 21 12:30:00.080388 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:30:00.080367    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781\": container with ID starting with c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781 not found: ID does not exist" containerID="c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781"
Jul 21 12:30:00.080433 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.080397    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781"} err="failed to get container status \"c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781\": rpc error: code = NotFound desc = could not find container \"c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781\": container with ID starting with c1cda94cb7eae339185e524c6bb73500bde843b40984d4d2cc425b530d44c781 not found: ID does not exist"
Jul 21 12:30:00.091839 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.091817    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7966fbf488-mb5tq"]
Jul 21 12:30:00.099121 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.099093    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7966fbf488-mb5tq"]
Jul 21 12:30:00.144004 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.143976    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh"]
Jul 21 12:30:00.144302 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.144289    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bbb11a5-38db-4812-93c7-76ce24416fb1" containerName="manager"
Jul 21 12:30:00.144368 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.144304    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bbb11a5-38db-4812-93c7-76ce24416fb1" containerName="manager"
Jul 21 12:30:00.144368 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.144361    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bbb11a5-38db-4812-93c7-76ce24416fb1" containerName="manager"
Jul 21 12:30:00.148357 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.148341    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh"
Jul 21 12:30:00.150594 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.150575    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\""
Jul 21 12:30:00.154316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.154291    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh"]
Jul 21 12:30:00.165350 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.165326    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh"
Jul 21 12:30:00.301415 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.301388    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh"]
Jul 21 12:30:00.302786 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:30:00.302745    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b240fbd_962c_4c4c_b2f9_03dfb124b536.slice/crio-1d10856d44838f396e0e89b3171f65903c555933c1a049ecdf81cb1fce6697a7 WatchSource:0}: Error finding container 1d10856d44838f396e0e89b3171f65903c555933c1a049ecdf81cb1fce6697a7: Status 404 returned error can't find the container with id 1d10856d44838f396e0e89b3171f65903c555933c1a049ecdf81cb1fce6697a7
Jul 21 12:30:00.416691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.416658    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bbb11a5-38db-4812-93c7-76ce24416fb1" path="/var/lib/kubelet/pods/7bbb11a5-38db-4812-93c7-76ce24416fb1/volumes"
Jul 21 12:30:00.865393 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.865358    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7966fbf488-xd7nf"]
Jul 21 12:30:00.869917 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.869897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:00.873251 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.873228    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 12:30:00.873366 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.873271    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hc8m7\""
Jul 21 12:30:00.873366 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.873298    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 12:30:00.881163 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.881142    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7966fbf488-xd7nf"]
Jul 21 12:30:00.969840 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.969802    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/587525a2-6503-413c-9b40-fbe28c8b41c4-webhook-cert\") pod \"maas-controller-7966fbf488-xd7nf\" (UID: \"587525a2-6503-413c-9b40-fbe28c8b41c4\") " pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:00.969840 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:00.969839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sh6s\" (UniqueName: \"kubernetes.io/projected/587525a2-6503-413c-9b40-fbe28c8b41c4-kube-api-access-9sh6s\") pod \"maas-controller-7966fbf488-xd7nf\" (UID: \"587525a2-6503-413c-9b40-fbe28c8b41c4\") " pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:01.070213 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:01.070180    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/587525a2-6503-413c-9b40-fbe28c8b41c4-webhook-cert\") pod \"maas-controller-7966fbf488-xd7nf\" (UID: \"587525a2-6503-413c-9b40-fbe28c8b41c4\") " pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:01.070213 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:01.070213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sh6s\" (UniqueName: \"kubernetes.io/projected/587525a2-6503-413c-9b40-fbe28c8b41c4-kube-api-access-9sh6s\") pod \"maas-controller-7966fbf488-xd7nf\" (UID: \"587525a2-6503-413c-9b40-fbe28c8b41c4\") " pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:01.073179 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:01.073152    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/587525a2-6503-413c-9b40-fbe28c8b41c4-webhook-cert\") pod \"maas-controller-7966fbf488-xd7nf\" (UID: \"587525a2-6503-413c-9b40-fbe28c8b41c4\") " pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:01.077991 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:01.077956    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh" event={"ID":"4b240fbd-962c-4c4c-b2f9-03dfb124b536","Type":"ContainerStarted","Data":"1d10856d44838f396e0e89b3171f65903c555933c1a049ecdf81cb1fce6697a7"}
Jul 21 12:30:01.078941 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:01.078918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sh6s\" (UniqueName: \"kubernetes.io/projected/587525a2-6503-413c-9b40-fbe28c8b41c4-kube-api-access-9sh6s\") pod \"maas-controller-7966fbf488-xd7nf\" (UID: \"587525a2-6503-413c-9b40-fbe28c8b41c4\") " pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:01.180688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:01.180602    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:01.332485 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:01.332433    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7966fbf488-xd7nf"]
Jul 21 12:30:01.335168 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:30:01.335141    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod587525a2_6503_413c_9b40_fbe28c8b41c4.slice/crio-16fdcbbabae2562885bab6ee3c1536fa4bc8ab76db7f7d67fc8edd3b89f94ef9 WatchSource:0}: Error finding container 16fdcbbabae2562885bab6ee3c1536fa4bc8ab76db7f7d67fc8edd3b89f94ef9: Status 404 returned error can't find the container with id 16fdcbbabae2562885bab6ee3c1536fa4bc8ab76db7f7d67fc8edd3b89f94ef9
Jul 21 12:30:02.086630 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:02.086592    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-xd7nf" event={"ID":"587525a2-6503-413c-9b40-fbe28c8b41c4","Type":"ContainerStarted","Data":"16fdcbbabae2562885bab6ee3c1536fa4bc8ab76db7f7d67fc8edd3b89f94ef9"}
Jul 21 12:30:03.092288 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:03.092257    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh" event={"ID":"4b240fbd-962c-4c4c-b2f9-03dfb124b536","Type":"ContainerStarted","Data":"5ead21e2e5849aeaf6dbcc6e12e6ab6dd9b25f955ec5509c7cdf96ca315beb8b"}
Jul 21 12:30:03.093750 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:03.093725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-xd7nf" event={"ID":"587525a2-6503-413c-9b40-fbe28c8b41c4","Type":"ContainerStarted","Data":"7354dc17b84f75ef6e006af34db1f663ea806cb8e26f87b44e2e7633594e0217"}
Jul 21 12:30:03.093885 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:03.093795    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:03.108747 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:03.108663    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh" podStartSLOduration=0.86233034 podStartE2EDuration="3.108648408s" totalImagesPullingTime="2.246318068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:30:03.10752416 +0000 UTC m=+1127.165955165" watchObservedRunningTime="2026-07-21 12:30:03.108648408 +0000 UTC m=+1127.167079412"
Jul 21 12:30:03.123881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:03.123833    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7966fbf488-xd7nf" podStartSLOduration=1.909824752 podStartE2EDuration="3.12381717s" totalImagesPullingTime="1.213992418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:30:03.122333961 +0000 UTC m=+1127.180764965" watchObservedRunningTime="2026-07-21 12:30:03.12381717 +0000 UTC m=+1127.182248166"
Jul 21 12:30:04.099550 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:04.099514    2562 generic.go:356] "Generic (PLEG): container finished" podID="4b240fbd-962c-4c4c-b2f9-03dfb124b536" containerID="5ead21e2e5849aeaf6dbcc6e12e6ab6dd9b25f955ec5509c7cdf96ca315beb8b" exitCode=0
Jul 21 12:30:04.099978 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:04.099601    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh" event={"ID":"4b240fbd-962c-4c4c-b2f9-03dfb124b536","Type":"ContainerDied","Data":"5ead21e2e5849aeaf6dbcc6e12e6ab6dd9b25f955ec5509c7cdf96ca315beb8b"}
Jul 21 12:30:05.240155 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:05.240130    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh"
Jul 21 12:30:06.111009 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:06.110976    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh"
Jul 21 12:30:06.111009 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:06.110992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743950-lsvbh" event={"ID":"4b240fbd-962c-4c4c-b2f9-03dfb124b536","Type":"ContainerDied","Data":"1d10856d44838f396e0e89b3171f65903c555933c1a049ecdf81cb1fce6697a7"}
Jul 21 12:30:06.111267 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:06.111035    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d10856d44838f396e0e89b3171f65903c555933c1a049ecdf81cb1fce6697a7"
Jul 21 12:30:14.101716 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:14.101681    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7966fbf488-xd7nf"
Jul 21 12:30:24.495281 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:24.495238    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:30:35.490676 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:30:35.490629    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:31:09.495447 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:31:09.495407    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:31:19.099150 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:31:19.099114    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:31:29.496943 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:31:29.496904    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:31:35.627097 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:31:35.627061    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:31:45.614684 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:31:45.614646    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:31:54.897552 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:31:54.897515    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:32:06.695383 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:32:06.695344    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:32:16.497354 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:32:16.497261    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:32:39.004011 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:32:39.003975    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:32:51.106463 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:32:51.106425    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:33:14.702957 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:33:14.702925    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:33:24.888905 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:33:24.888868    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:33:40.906237 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:33:40.906154    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:33:50.606417 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:33:50.606379    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:34:00.001035 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:34:00.000997    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:34:09.293965 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:34:09.293924    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:34:18.497413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:34:18.497374    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:36:06.387818 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:36:06.387782    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:36:15.399297 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:36:15.399263    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:36:28.793011 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:36:28.792974    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:36:39.992564 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:36:39.992486    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:36:50.503609 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:36:50.503571    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:37:01.508160 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:37:01.508120    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:37:11.197723 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:37:11.197688    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:37:22.122621 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:37:22.122587    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:37:31.900884 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:37:31.900845    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:37:39.516829 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:37:39.516736    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:37:48.702376 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:37:48.702338    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:37:58.002936 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:37:58.002902    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:38:20.602278 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:20.602199    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:38:30.399212 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:30.399172    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:38:35.995087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:35.995050    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"]
Jul 21 12:38:35.995448 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:35.995361    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b240fbd-962c-4c4c-b2f9-03dfb124b536" containerName="cleanup"
Jul 21 12:38:35.995448 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:35.995371    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b240fbd-962c-4c4c-b2f9-03dfb124b536" containerName="cleanup"
Jul 21 12:38:35.995448 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:35.995423    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b240fbd-962c-4c4c-b2f9-03dfb124b536" containerName="cleanup"
Jul 21 12:38:35.998494 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:35.998474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.001040 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.001017    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-794e69c5-kserve-self-signed-certs\""
Jul 21 12:38:36.019783 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.019752    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"]
Jul 21 12:38:36.159165 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.159125    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tls-certs\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.159165 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.159168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfjz8\" (UniqueName: \"kubernetes.io/projected/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-kube-api-access-kfjz8\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.159366 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.159240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-home\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.159366 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.159270    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tmp-dir\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.159435 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.159365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-dshm\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.159435 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.159413    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-model-cache\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.260483 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.260390    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-dshm\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.260483 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.260455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-model-cache\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.260704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.260491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tls-certs\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.260704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.260517    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfjz8\" (UniqueName: \"kubernetes.io/projected/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-kube-api-access-kfjz8\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.260704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.260554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-home\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.260704 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.260573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tmp-dir\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.260984 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.260946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-model-cache\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.261038 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.260981    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tmp-dir\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.261038 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.261023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-home\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.262873 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.262854    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-dshm\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.263127 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.263111    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tls-certs\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.269127 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.269101    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfjz8\" (UniqueName: \"kubernetes.io/projected/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-kube-api-access-kfjz8\") pod \"e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") " pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.307204 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.307170    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:36.464861 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.464823    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"]
Jul 21 12:38:36.466976 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:38:36.466949    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8ab6cc4_67d9_4755_b179_1bdc43a1cf50.slice/crio-6b3d922f55759464efafa90a170d0fe205c909706860d224a958b188b3c9dbda WatchSource:0}: Error finding container 6b3d922f55759464efafa90a170d0fe205c909706860d224a958b188b3c9dbda: Status 404 returned error can't find the container with id 6b3d922f55759464efafa90a170d0fe205c909706860d224a958b188b3c9dbda
Jul 21 12:38:36.558259 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.558222    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj" event={"ID":"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50","Type":"ContainerStarted","Data":"964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238"}
Jul 21 12:38:36.558259 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.558265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj" event={"ID":"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50","Type":"ContainerStarted","Data":"6b3d922f55759464efafa90a170d0fe205c909706860d224a958b188b3c9dbda"}
Jul 21 12:38:36.578381 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:36.578327    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj" podStartSLOduration=1.57830793 podStartE2EDuration="1.57830793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:38:36.578112996 +0000 UTC m=+1640.636544000" watchObservedRunningTime="2026-07-21 12:38:36.57830793 +0000 UTC m=+1640.636738934"
Jul 21 12:38:41.799664 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:41.799628    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:38:46.308281 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:46.308187    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:46.308281 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:46.308246    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:46.321179 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:46.321153    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:38:46.617974 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:38:46.617950    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:39:15.551418 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:15.551387    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"]
Jul 21 12:39:19.493601 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:19.493561    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:39:30.572677 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.572611    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj" podUID="a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" containerName="main" containerID="cri-o://964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238" gracePeriod=45
Jul 21 12:39:30.816536 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.816513    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:39:30.818454 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.818427    2562 generic.go:356] "Generic (PLEG): container finished" podID="a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" containerID="964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238" exitCode=0
Jul 21 12:39:30.818558 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.818462    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"
Jul 21 12:39:30.818558 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.818470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj" event={"ID":"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50","Type":"ContainerDied","Data":"964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238"}
Jul 21 12:39:30.818558 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.818501    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj" event={"ID":"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50","Type":"ContainerDied","Data":"6b3d922f55759464efafa90a170d0fe205c909706860d224a958b188b3c9dbda"}
Jul 21 12:39:30.818558 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.818517    2562 scope.go:117] "RemoveContainer" containerID="964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238"
Jul 21 12:39:30.826785 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.826752    2562 scope.go:117] "RemoveContainer" containerID="964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238"
Jul 21 12:39:30.827063 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:39:30.827040    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238\": container with ID starting with 964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238 not found: ID does not exist" containerID="964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238"
Jul 21 12:39:30.827157 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.827069    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238"} err="failed to get container status \"964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238\": rpc error: code = NotFound desc = could not find container \"964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238\": container with ID starting with 964c6ae0340ac7a9f7a44f15573e35fdfae4cf5bc619fb06274af2bece5c4238 not found: ID does not exist"
Jul 21 12:39:30.937562 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.937526    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tmp-dir\") pod \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") "
Jul 21 12:39:30.937562 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.937572    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-model-cache\") pod \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") "
Jul 21 12:39:30.937834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.937610    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-dshm\") pod \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") "
Jul 21 12:39:30.937834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.937631    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-home\") pod \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") "
Jul 21 12:39:30.937834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.937652    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tls-certs\") pod \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") "
Jul 21 12:39:30.937834 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.937709    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfjz8\" (UniqueName: \"kubernetes.io/projected/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-kube-api-access-kfjz8\") pod \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\" (UID: \"a8ab6cc4-67d9-4755-b179-1bdc43a1cf50\") "
Jul 21 12:39:30.938046 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.937847    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" (UID: "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:39:30.938046 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.937882    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" (UID: "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:39:30.938046 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.937942    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-home" (OuterVolumeSpecName: "home") pod "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" (UID: "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:39:30.938168 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.938052    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-home\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:39:30.938168 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.938071    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tmp-dir\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:39:30.938168 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.938088    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-model-cache\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:39:30.940015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.939989    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-dshm" (OuterVolumeSpecName: "dshm") pod "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" (UID: "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:39:30.940126 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.940037    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" (UID: "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:39:30.940126 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:30.940056    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-kube-api-access-kfjz8" (OuterVolumeSpecName: "kube-api-access-kfjz8") pod "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" (UID: "a8ab6cc4-67d9-4755-b179-1bdc43a1cf50"). InnerVolumeSpecName "kube-api-access-kfjz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:39:31.039106 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:31.039054    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-dshm\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:39:31.039106 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:31.039100    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-tls-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:39:31.039106 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:31.039111    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfjz8\" (UniqueName: \"kubernetes.io/projected/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50-kube-api-access-kfjz8\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:39:31.140324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:31.140297    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"]
Jul 21 12:39:31.144016 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:31.143989    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-794e69c5-kserve-58bd968959-db2kj"]
Jul 21 12:39:32.416331 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:32.416301    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" path="/var/lib/kubelet/pods/a8ab6cc4-67d9-4755-b179-1bdc43a1cf50/volumes"
Jul 21 12:39:32.789040 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:32.788953    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:39:39.994201 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:39.994113    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:39:54.189925 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:39:54.189884    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:40:05.703130 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:40:05.703096    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:40:15.089861 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:40:15.089825    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:40:25.298510 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:40:25.298470    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:40:34.592841 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:40:34.592809    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:40:43.892157 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:40:43.892117    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:40:56.601813 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:40:56.601762    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:41:05.997292 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:41:05.997257    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:41:15.500133 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:41:15.500092    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:41:25.394980 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:41:25.394946    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:41:37.096433 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:41:37.096397    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:41:46.393438 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:41:46.393393    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:41:58.101571 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:41:58.101530    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:42:10.093976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:10.093939    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:42:20.092817 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:20.092756    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:42:33.892386 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:33.892338    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:42:43.690277 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:43.690195    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:42:53.095943 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.095896    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:42:53.213577 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.213544    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"]
Jul 21 12:42:53.213936 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.213920    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" containerName="main"
Jul 21 12:42:53.214029 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.213940    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" containerName="main"
Jul 21 12:42:53.214083 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.214046    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8ab6cc4-67d9-4755-b179-1bdc43a1cf50" containerName="main"
Jul 21 12:42:53.217080 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.217058    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.221185 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.221160    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9fb423b2-openshift-default-dockercfg-4qp72\""
Jul 21 12:42:53.244291 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.244261    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"]
Jul 21 12:42:53.287724 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.287682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-data\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.287915 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.287742    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7919d723-adce-44aa-b544-7582095d7f19-istiod-ca-cert\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.287915 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.287810    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67hgc\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-kube-api-access-67hgc\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.287915 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.287835    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-credential-socket\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.287915 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.287872    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-socket\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.287915 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.287903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-certs\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.288130 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.287952    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7919d723-adce-44aa-b544-7582095d7f19-istio-podinfo\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.288130 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.287967    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-istio-token\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.288130 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.287993    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-envoy\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.388894 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.388806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-socket\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.388894 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.388847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-certs\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.388894 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.388876    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7919d723-adce-44aa-b544-7582095d7f19-istio-podinfo\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.388894 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.388891    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-istio-token\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389235 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.388914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-envoy\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389235 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.388947    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-data\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389235 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.388970    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7919d723-adce-44aa-b544-7582095d7f19-istiod-ca-cert\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389235 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.389011    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67hgc\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-kube-api-access-67hgc\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389235 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.389035    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-credential-socket\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389444 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.389285    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-socket\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389501 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.389489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-certs\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389603 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.389571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-data\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389738 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.389600    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-credential-socket\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.389818 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.389736    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7919d723-adce-44aa-b544-7582095d7f19-istiod-ca-cert\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.391553 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.391535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-envoy\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.391633 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.391579    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7919d723-adce-44aa-b544-7582095d7f19-istio-podinfo\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.397064 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.397040    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67hgc\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-kube-api-access-67hgc\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.397567 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.397540    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-istio-token\") pod \"e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") " pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.527514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.527481    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:53.661242 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.661166    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"]
Jul 21 12:42:53.664025 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:42:53.663985    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7919d723_adce_44aa_b544_7582095d7f19.slice/crio-c7388b121140a51bc267d961d1f3be225651e22dfa154bbf40358993a2ed97aa WatchSource:0}: Error finding container c7388b121140a51bc267d961d1f3be225651e22dfa154bbf40358993a2ed97aa: Status 404 returned error can't find the container with id c7388b121140a51bc267d961d1f3be225651e22dfa154bbf40358993a2ed97aa
Jul 21 12:42:53.666038 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.666021    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:42:53.803459 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:53.803422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" event={"ID":"7919d723-adce-44aa-b544-7582095d7f19","Type":"ContainerStarted","Data":"c7388b121140a51bc267d961d1f3be225651e22dfa154bbf40358993a2ed97aa"}
Jul 21 12:42:56.352650 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:56.352608    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:42:56.352949 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:56.352691    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:42:56.352949 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:56.352718    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:42:56.820516 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:56.820484    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" event={"ID":"7919d723-adce-44aa-b544-7582095d7f19","Type":"ContainerStarted","Data":"88edb5a1b8d6f7ca24668279f69c916c2590e5c06b3bfd012da8d462432360a9"}
Jul 21 12:42:56.843210 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:56.843144    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" podStartSLOduration=1.156987771 podStartE2EDuration="3.843128299s" totalImagesPullingTime="2.686140528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:42:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:42:56.841201799 +0000 UTC m=+1900.899632830" watchObservedRunningTime="2026-07-21 12:42:56.843128299 +0000 UTC m=+1900.901559358"
Jul 21 12:42:57.528159 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:57.528119    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:57.533468 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:57.533438    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:57.825934 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:57.825900    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:57.826893 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:57.826870    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:42:58.296721 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.296643    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"]
Jul 21 12:42:58.300292 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.300264    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.303084 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.303054    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:42:58.303084 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.303072    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\""
Jul 21 12:42:58.303913 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.303895    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9fb423b2-serving-cert\""
Jul 21 12:42:58.304052 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.304031    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:42:58.313829 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.313806    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"]
Jul 21 12:42:58.435447 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.435405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-maas-api-tls\") pod \"maas-api-e2e-ait-9fb423b2-64f847b59b-s747h\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.435628 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.435454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrtvh\" (UniqueName: \"kubernetes.io/projected/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-kube-api-access-lrtvh\") pod \"maas-api-e2e-ait-9fb423b2-64f847b59b-s747h\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.435628 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.435559    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-openshift-service-ca\") pod \"maas-api-e2e-ait-9fb423b2-64f847b59b-s747h\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.536743 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.536707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-openshift-service-ca\") pod \"maas-api-e2e-ait-9fb423b2-64f847b59b-s747h\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.537230 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.536765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-maas-api-tls\") pod \"maas-api-e2e-ait-9fb423b2-64f847b59b-s747h\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.537230 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.536833    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrtvh\" (UniqueName: \"kubernetes.io/projected/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-kube-api-access-lrtvh\") pod \"maas-api-e2e-ait-9fb423b2-64f847b59b-s747h\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.537345 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.537331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-openshift-service-ca\") pod \"maas-api-e2e-ait-9fb423b2-64f847b59b-s747h\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.539564 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.539533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-maas-api-tls\") pod \"maas-api-e2e-ait-9fb423b2-64f847b59b-s747h\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.549581 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.549514    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrtvh\" (UniqueName: \"kubernetes.io/projected/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-kube-api-access-lrtvh\") pod \"maas-api-e2e-ait-9fb423b2-64f847b59b-s747h\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.611053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.611003    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:42:58.744888 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.744862    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"]
Jul 21 12:42:58.747926 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:42:58.747898    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e68abca_8cd3_4bb1_be4d_01116cf2b4f1.slice/crio-9b9275b29daa798f00d10b2937d9516be85511d26b912ade4dfd2e713d1fe3b0 WatchSource:0}: Error finding container 9b9275b29daa798f00d10b2937d9516be85511d26b912ade4dfd2e713d1fe3b0: Status 404 returned error can't find the container with id 9b9275b29daa798f00d10b2937d9516be85511d26b912ade4dfd2e713d1fe3b0
Jul 21 12:42:58.832080 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:42:58.832047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h" event={"ID":"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1","Type":"ContainerStarted","Data":"9b9275b29daa798f00d10b2937d9516be85511d26b912ade4dfd2e713d1fe3b0"}
Jul 21 12:43:00.844411 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:00.844372    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h" event={"ID":"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1","Type":"ContainerStarted","Data":"85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887"}
Jul 21 12:43:00.844896 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:00.844432    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:43:00.868138 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:00.868077    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h" podStartSLOduration=1.3725221140000001 podStartE2EDuration="2.868062331s" totalImagesPullingTime="1.495540217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:43:00.866323597 +0000 UTC m=+1904.924754625" watchObservedRunningTime="2026-07-21 12:43:00.868062331 +0000 UTC m=+1904.926493334"
Jul 21 12:43:01.800766 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:01.800730    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:43:06.863690 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:06.863655    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:43:09.891621 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:09.891534    2562 generic.go:356] "Generic (PLEG): container finished" podID="7919d723-adce-44aa-b544-7582095d7f19" containerID="88edb5a1b8d6f7ca24668279f69c916c2590e5c06b3bfd012da8d462432360a9" exitCode=0
Jul 21 12:43:09.892147 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:09.891616    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" event={"ID":"7919d723-adce-44aa-b544-7582095d7f19","Type":"ContainerDied","Data":"88edb5a1b8d6f7ca24668279f69c916c2590e5c06b3bfd012da8d462432360a9"}
Jul 21 12:43:09.892147 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:09.892126    2562 scope.go:117] "RemoveContainer" containerID="88edb5a1b8d6f7ca24668279f69c916c2590e5c06b3bfd012da8d462432360a9"
Jul 21 12:43:09.894195 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:09.894157    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:43:09.894299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:09.894229    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:43:09.894299 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:09.894265    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:43:10.899296 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:10.899261    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" event={"ID":"7919d723-adce-44aa-b544-7582095d7f19","Type":"ContainerStarted","Data":"4533780cdb9d06f7e21ba199113609c3c7b904fd59a95dafb884e68b9087d60a"}
Jul 21 12:43:11.527963 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:11.527921    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:43:11.529172 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:11.529141    2562 patch_prober.go:28] interesting pod/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 21 12:43:11.529291 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:11.529195    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 21 12:43:12.527904 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:12.527865    2562 patch_prober.go:28] interesting pod/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 21 12:43:12.528274 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:12.527930    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 21 12:43:12.825923 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:12.825879    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:43:13.528669 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:13.528631    2562 patch_prober.go:28] interesting pod/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 21 12:43:13.529074 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:13.528693    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 21 12:43:14.528574 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:14.528531    2562 patch_prober.go:28] interesting pod/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 21 12:43:14.528803 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:14.528597    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 21 12:43:15.531996 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:15.531956    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:43:15.532999 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:15.532979    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:43:34.119039 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.119005    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"]
Jul 21 12:43:34.119597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.119308    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h" podUID="6e68abca-8cd3-4bb1-be4d-01116cf2b4f1" containerName="maas-api" containerID="cri-o://85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887" gracePeriod=30
Jul 21 12:43:34.372829 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.372754    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:43:34.448049 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.448023    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrtvh\" (UniqueName: \"kubernetes.io/projected/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-kube-api-access-lrtvh\") pod \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") "
Jul 21 12:43:34.448204 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.448072    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-openshift-service-ca\") pod \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") "
Jul 21 12:43:34.448204 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.448149    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-maas-api-tls\") pod \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\" (UID: \"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1\") "
Jul 21 12:43:34.448494 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.448469    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e68abca-8cd3-4bb1-be4d-01116cf2b4f1" (UID: "6e68abca-8cd3-4bb1-be4d-01116cf2b4f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:43:34.450443 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.450415    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e68abca-8cd3-4bb1-be4d-01116cf2b4f1" (UID: "6e68abca-8cd3-4bb1-be4d-01116cf2b4f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:43:34.450548 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.450490    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-kube-api-access-lrtvh" (OuterVolumeSpecName: "kube-api-access-lrtvh") pod "6e68abca-8cd3-4bb1-be4d-01116cf2b4f1" (UID: "6e68abca-8cd3-4bb1-be4d-01116cf2b4f1"). InnerVolumeSpecName "kube-api-access-lrtvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:43:34.548685 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.548649    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:34.548685 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.548677    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrtvh\" (UniqueName: \"kubernetes.io/projected/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-kube-api-access-lrtvh\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:34.548685 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:34.548688    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:35.016017 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:35.015979    2562 generic.go:356] "Generic (PLEG): container finished" podID="6e68abca-8cd3-4bb1-be4d-01116cf2b4f1" containerID="85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887" exitCode=0
Jul 21 12:43:35.016017 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:35.016012    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"
Jul 21 12:43:35.016239 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:35.016058    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h" event={"ID":"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1","Type":"ContainerDied","Data":"85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887"}
Jul 21 12:43:35.016239 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:35.016103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h" event={"ID":"6e68abca-8cd3-4bb1-be4d-01116cf2b4f1","Type":"ContainerDied","Data":"9b9275b29daa798f00d10b2937d9516be85511d26b912ade4dfd2e713d1fe3b0"}
Jul 21 12:43:35.016239 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:35.016124    2562 scope.go:117] "RemoveContainer" containerID="85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887"
Jul 21 12:43:35.025117 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:35.025098    2562 scope.go:117] "RemoveContainer" containerID="85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887"
Jul 21 12:43:35.025383 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:43:35.025359    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887\": container with ID starting with 85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887 not found: ID does not exist" containerID="85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887"
Jul 21 12:43:35.025443 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:35.025395    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887"} err="failed to get container status \"85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887\": rpc error: code = NotFound desc = could not find container \"85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887\": container with ID starting with 85bfc8c9b65eee8df476759e3c1fa92f144206c9c334720a90ee7265a342e887 not found: ID does not exist"
Jul 21 12:43:35.037440 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:35.037413    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"]
Jul 21 12:43:35.041407 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:35.041387    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9fb423b2-64f847b59b-s747h"]
Jul 21 12:43:36.417198 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:36.417164    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e68abca-8cd3-4bb1-be4d-01116cf2b4f1" path="/var/lib/kubelet/pods/6e68abca-8cd3-4bb1-be4d-01116cf2b4f1/volumes"
Jul 21 12:43:37.600999 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:37.600963    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:43:38.921892 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:38.921855    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"]
Jul 21 12:43:38.922326 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:38.922077    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy" containerID="cri-o://4533780cdb9d06f7e21ba199113609c3c7b904fd59a95dafb884e68b9087d60a" gracePeriod=30
Jul 21 12:43:39.946885 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.946848    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"]
Jul 21 12:43:39.947356 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.947229    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e68abca-8cd3-4bb1-be4d-01116cf2b4f1" containerName="maas-api"
Jul 21 12:43:39.947356 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.947247    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e68abca-8cd3-4bb1-be4d-01116cf2b4f1" containerName="maas-api"
Jul 21 12:43:39.947356 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.947325    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e68abca-8cd3-4bb1-be4d-01116cf2b4f1" containerName="maas-api"
Jul 21 12:43:39.951609 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.951588    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:39.954245 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.954220    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-21bec010-gw-openshift-default-dockercfg-xrh48\""
Jul 21 12:43:39.963275 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.963248    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"]
Jul 21 12:43:39.986993 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.986963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-podinfo\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:39.987159 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.987078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2rwf\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-kube-api-access-h2rwf\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:39.987242 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.987210    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f77eed-5673-49cd-93f3-fce9fd398eb9-istiod-ca-cert\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:39.987307 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.987268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-credential-socket\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:39.987361 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.987309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-socket\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:39.987414 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.987372    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-token\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:39.987470 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.987429    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-data\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:39.987522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.987497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-envoy\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:39.987576 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:39.987549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-certs\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.088607 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.088571    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-envoy\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.088826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.088616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-certs\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.088826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.088651    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-podinfo\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.088826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.088677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2rwf\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-kube-api-access-h2rwf\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.088826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.088699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f77eed-5673-49cd-93f3-fce9fd398eb9-istiod-ca-cert\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.088826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.088729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-credential-socket\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.088826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.088757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-socket\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.089154 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.088924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-token\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.089154 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.089002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-data\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.089154 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.089077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-certs\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.089316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.089295    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-socket\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.089380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.089340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-data\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.089562 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.089495    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f77eed-5673-49cd-93f3-fce9fd398eb9-istiod-ca-cert\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.089685 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.089617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-credential-socket\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.091186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.091155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-envoy\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.091543 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.091527    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-podinfo\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.096729 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.096705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-token\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.097029 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.097008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2rwf\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-kube-api-access-h2rwf\") pod \"e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") " pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.263235 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.263145    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:40.401367 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.401337    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"]
Jul 21 12:43:40.404469 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:43:40.404441    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f77eed_5673_49cd_93f3_fce9fd398eb9.slice/crio-482506c569f305e3699f5105bb85835ea88e75c65dcb074f21af5270b7df8155 WatchSource:0}: Error finding container 482506c569f305e3699f5105bb85835ea88e75c65dcb074f21af5270b7df8155: Status 404 returned error can't find the container with id 482506c569f305e3699f5105bb85835ea88e75c65dcb074f21af5270b7df8155
Jul 21 12:43:40.406658 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.406627    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:43:40.406727 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.406688    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:43:40.406784 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:40.406724    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:43:41.050722 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:41.050684    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" event={"ID":"79f77eed-5673-49cd-93f3-fce9fd398eb9","Type":"ContainerStarted","Data":"b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8"}
Jul 21 12:43:41.051238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:41.050743    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" event={"ID":"79f77eed-5673-49cd-93f3-fce9fd398eb9","Type":"ContainerStarted","Data":"482506c569f305e3699f5105bb85835ea88e75c65dcb074f21af5270b7df8155"}
Jul 21 12:43:41.071276 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:41.071212    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" podStartSLOduration=2.071190181 podStartE2EDuration="2.071190181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:43:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:43:41.070582034 +0000 UTC m=+1945.129013073" watchObservedRunningTime="2026-07-21 12:43:41.071190181 +0000 UTC m=+1945.129621186"
Jul 21 12:43:41.263793 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:41.263706    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:41.268574 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:41.268544    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:42.056338 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.056309    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:42.057422 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.057400    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:42.609672 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.609635    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"]
Jul 21 12:43:42.613473 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.613449    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:42.616042 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.616017    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-21bec010-serving-cert\""
Jul 21 12:43:42.616180 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.616043    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:43:42.616934 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.616911    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\""
Jul 21 12:43:42.617060 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.616933    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:43:42.624295 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.624271    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"]
Jul 21 12:43:42.710107 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.710062    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e5377e-dedf-4cbe-8fed-f6e194f72341-openshift-service-ca\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:42.710305 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.710133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55e5377e-dedf-4cbe-8fed-f6e194f72341-maas-api-tls\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:42.710305 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.710171    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nxwt\" (UniqueName: \"kubernetes.io/projected/55e5377e-dedf-4cbe-8fed-f6e194f72341-kube-api-access-4nxwt\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:42.811329 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.811281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e5377e-dedf-4cbe-8fed-f6e194f72341-openshift-service-ca\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:42.811535 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.811365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55e5377e-dedf-4cbe-8fed-f6e194f72341-maas-api-tls\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:42.811535 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.811396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nxwt\" (UniqueName: \"kubernetes.io/projected/55e5377e-dedf-4cbe-8fed-f6e194f72341-kube-api-access-4nxwt\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:42.811654 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:43:42.811531    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-serving-cert: secret "maas-api-e2e-migrate-21bec010-serving-cert" not found
Jul 21 12:43:42.811654 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:43:42.811606    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e5377e-dedf-4cbe-8fed-f6e194f72341-maas-api-tls podName:55e5377e-dedf-4cbe-8fed-f6e194f72341 nodeName:}" failed. No retries permitted until 2026-07-21 12:43:43.311584693 +0000 UTC m=+1947.370015695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/55e5377e-dedf-4cbe-8fed-f6e194f72341-maas-api-tls") pod "maas-api-e2e-migrate-21bec010-db9559655-ckdpt" (UID: "55e5377e-dedf-4cbe-8fed-f6e194f72341") : secret "maas-api-e2e-migrate-21bec010-serving-cert" not found
Jul 21 12:43:42.812056 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.812038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e5377e-dedf-4cbe-8fed-f6e194f72341-openshift-service-ca\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:42.820836 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:42.820812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nxwt\" (UniqueName: \"kubernetes.io/projected/55e5377e-dedf-4cbe-8fed-f6e194f72341-kube-api-access-4nxwt\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:43.316452 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:43.316413    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55e5377e-dedf-4cbe-8fed-f6e194f72341-maas-api-tls\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:43.318969 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:43.318947    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55e5377e-dedf-4cbe-8fed-f6e194f72341-maas-api-tls\") pod \"maas-api-e2e-migrate-21bec010-db9559655-ckdpt\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:43.525726 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:43.525689    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:43.667554 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:43.667527    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"]
Jul 21 12:43:43.669056 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:43:43.669029    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e5377e_dedf_4cbe_8fed_f6e194f72341.slice/crio-8e3f40d51963fc3c771853b8e85a3b45326e4dfb313754efae6c42327fd47cc1 WatchSource:0}: Error finding container 8e3f40d51963fc3c771853b8e85a3b45326e4dfb313754efae6c42327fd47cc1: Status 404 returned error can't find the container with id 8e3f40d51963fc3c771853b8e85a3b45326e4dfb313754efae6c42327fd47cc1
Jul 21 12:43:44.069125 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.069086    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt" event={"ID":"55e5377e-dedf-4cbe-8fed-f6e194f72341","Type":"ContainerStarted","Data":"8e3f40d51963fc3c771853b8e85a3b45326e4dfb313754efae6c42327fd47cc1"}
Jul 21 12:43:44.071231 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.071203    2562 generic.go:356] "Generic (PLEG): container finished" podID="7919d723-adce-44aa-b544-7582095d7f19" containerID="4533780cdb9d06f7e21ba199113609c3c7b904fd59a95dafb884e68b9087d60a" exitCode=0
Jul 21 12:43:44.071370 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.071285    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" event={"ID":"7919d723-adce-44aa-b544-7582095d7f19","Type":"ContainerDied","Data":"4533780cdb9d06f7e21ba199113609c3c7b904fd59a95dafb884e68b9087d60a"}
Jul 21 12:43:44.071370 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.071341    2562 scope.go:117] "RemoveContainer" containerID="88edb5a1b8d6f7ca24668279f69c916c2590e5c06b3bfd012da8d462432360a9"
Jul 21 12:43:44.181079 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.181055    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:43:44.224901 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.224825    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-socket\") pod \"7919d723-adce-44aa-b544-7582095d7f19\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") "
Jul 21 12:43:44.224901 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.224861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-istio-token\") pod \"7919d723-adce-44aa-b544-7582095d7f19\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") "
Jul 21 12:43:44.224901 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.224884    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-data\") pod \"7919d723-adce-44aa-b544-7582095d7f19\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") "
Jul 21 12:43:44.225164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.224906    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-credential-socket\") pod \"7919d723-adce-44aa-b544-7582095d7f19\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") "
Jul 21 12:43:44.225164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.224954    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7919d723-adce-44aa-b544-7582095d7f19-istiod-ca-cert\") pod \"7919d723-adce-44aa-b544-7582095d7f19\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") "
Jul 21 12:43:44.225164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225014    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67hgc\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-kube-api-access-67hgc\") pod \"7919d723-adce-44aa-b544-7582095d7f19\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") "
Jul 21 12:43:44.225164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225049    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7919d723-adce-44aa-b544-7582095d7f19-istio-podinfo\") pod \"7919d723-adce-44aa-b544-7582095d7f19\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") "
Jul 21 12:43:44.225164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225081    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-envoy\") pod \"7919d723-adce-44aa-b544-7582095d7f19\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") "
Jul 21 12:43:44.225164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225109    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-certs\") pod \"7919d723-adce-44aa-b544-7582095d7f19\" (UID: \"7919d723-adce-44aa-b544-7582095d7f19\") "
Jul 21 12:43:44.225452 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225205    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7919d723-adce-44aa-b544-7582095d7f19" (UID: "7919d723-adce-44aa-b544-7582095d7f19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:43:44.225452 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225269    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7919d723-adce-44aa-b544-7582095d7f19" (UID: "7919d723-adce-44aa-b544-7582095d7f19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:43:44.225452 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225399    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7919d723-adce-44aa-b544-7582095d7f19" (UID: "7919d723-adce-44aa-b544-7582095d7f19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:43:44.225452 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225425    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:44.225452 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225445    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:44.225694 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225575    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7919d723-adce-44aa-b544-7582095d7f19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7919d723-adce-44aa-b544-7582095d7f19" (UID: "7919d723-adce-44aa-b544-7582095d7f19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:43:44.225749 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.225688    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-data" (OuterVolumeSpecName: "istio-data") pod "7919d723-adce-44aa-b544-7582095d7f19" (UID: "7919d723-adce-44aa-b544-7582095d7f19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:43:44.227357 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.227327    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7919d723-adce-44aa-b544-7582095d7f19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7919d723-adce-44aa-b544-7582095d7f19" (UID: "7919d723-adce-44aa-b544-7582095d7f19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:43:44.227357 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.227337    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-istio-token" (OuterVolumeSpecName: "istio-token") pod "7919d723-adce-44aa-b544-7582095d7f19" (UID: "7919d723-adce-44aa-b544-7582095d7f19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:43:44.227503 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.227361    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-kube-api-access-67hgc" (OuterVolumeSpecName: "kube-api-access-67hgc") pod "7919d723-adce-44aa-b544-7582095d7f19" (UID: "7919d723-adce-44aa-b544-7582095d7f19"). InnerVolumeSpecName "kube-api-access-67hgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:43:44.227503 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.227441    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7919d723-adce-44aa-b544-7582095d7f19" (UID: "7919d723-adce-44aa-b544-7582095d7f19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:43:44.326316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.326277    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:44.326316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.326309    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:44.326316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.326321    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7919d723-adce-44aa-b544-7582095d7f19-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:44.326811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.326334    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67hgc\" (UniqueName: \"kubernetes.io/projected/7919d723-adce-44aa-b544-7582095d7f19-kube-api-access-67hgc\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:44.326811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.326349    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7919d723-adce-44aa-b544-7582095d7f19-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:44.326811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.326360    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:44.326811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:44.326372    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7919d723-adce-44aa-b544-7582095d7f19-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:43:45.077816 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:45.077759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt" event={"ID":"55e5377e-dedf-4cbe-8fed-f6e194f72341","Type":"ContainerStarted","Data":"820772a11508460bb7b910fb79ca6eb944168bdf5070f796d3390ca5e8223b98"}
Jul 21 12:43:45.078072 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:45.077847    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:45.079011 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:45.078984    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"
Jul 21 12:43:45.079099 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:45.078988    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn" event={"ID":"7919d723-adce-44aa-b544-7582095d7f19","Type":"ContainerDied","Data":"c7388b121140a51bc267d961d1f3be225651e22dfa154bbf40358993a2ed97aa"}
Jul 21 12:43:45.079142 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:45.079111    2562 scope.go:117] "RemoveContainer" containerID="4533780cdb9d06f7e21ba199113609c3c7b904fd59a95dafb884e68b9087d60a"
Jul 21 12:43:45.097097 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:45.097054    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt" podStartSLOduration=2.770490183 podStartE2EDuration="3.097038088s" totalImagesPullingTime="326.547905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:43:45.095801835 +0000 UTC m=+1949.154232839" watchObservedRunningTime="2026-07-21 12:43:45.097038088 +0000 UTC m=+1949.155469092"
Jul 21 12:43:45.109124 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:45.109102    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"]
Jul 21 12:43:45.112628 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:45.112605    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9fb423b2-openshift-default-57b557cd7-9wpmn"]
Jul 21 12:43:46.416311 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:46.416274    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7919d723-adce-44aa-b544-7582095d7f19" path="/var/lib/kubelet/pods/7919d723-adce-44aa-b544-7582095d7f19/volumes"
Jul 21 12:43:48.892395 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:48.892340    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:43:51.090162 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:51.090130    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:43:57.057919 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:57.057825    2562 patch_prober.go:28] interesting pod/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 21 12:43:57.057919 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:57.057878    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 21 12:43:57.139197 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:57.139166    2562 generic.go:356] "Generic (PLEG): container finished" podID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerID="b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8" exitCode=0
Jul 21 12:43:57.139363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:57.139256    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" event={"ID":"79f77eed-5673-49cd-93f3-fce9fd398eb9","Type":"ContainerDied","Data":"b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8"}
Jul 21 12:43:57.139687 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:57.139670    2562 scope.go:117] "RemoveContainer" containerID="b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8"
Jul 21 12:43:57.140691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:57.140662    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:43:57.140822 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:57.140758    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:43:57.140893 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:57.140836    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:43:58.149489 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:58.149444    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" event={"ID":"79f77eed-5673-49cd-93f3-fce9fd398eb9","Type":"ContainerStarted","Data":"70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82"}
Jul 21 12:43:58.264073 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:58.264025    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:43:58.264298 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:58.264268    2562 patch_prober.go:28] interesting pod/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 21 12:43:58.264404 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:58.264332    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 21 12:43:59.263740 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:59.263690    2562 patch_prober.go:28] interesting pod/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 21 12:43:59.264283 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:43:59.263797    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 21 12:44:00.264391 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:00.264341    2562 patch_prober.go:28] interesting pod/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 21 12:44:00.264824 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:00.264410    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 21 12:44:01.263883 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:01.263845    2562 patch_prober.go:28] interesting pod/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 21 12:44:01.264066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:01.263905    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 21 12:44:02.267158 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:02.267128    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:44:02.267527 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:02.267399    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:44:02.268071 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:02.268054    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:44:18.061278 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.061243    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"]
Jul 21 12:44:18.061682 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.061453    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt" podUID="55e5377e-dedf-4cbe-8fed-f6e194f72341" containerName="maas-api" containerID="cri-o://820772a11508460bb7b910fb79ca6eb944168bdf5070f796d3390ca5e8223b98" gracePeriod=30
Jul 21 12:44:18.253086 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.253053    2562 generic.go:356] "Generic (PLEG): container finished" podID="55e5377e-dedf-4cbe-8fed-f6e194f72341" containerID="820772a11508460bb7b910fb79ca6eb944168bdf5070f796d3390ca5e8223b98" exitCode=0
Jul 21 12:44:18.253250 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.253127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt" event={"ID":"55e5377e-dedf-4cbe-8fed-f6e194f72341","Type":"ContainerDied","Data":"820772a11508460bb7b910fb79ca6eb944168bdf5070f796d3390ca5e8223b98"}
Jul 21 12:44:18.311818 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.311743    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:44:18.422885 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.422857    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e5377e-dedf-4cbe-8fed-f6e194f72341-openshift-service-ca\") pod \"55e5377e-dedf-4cbe-8fed-f6e194f72341\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") "
Jul 21 12:44:18.422885 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.422889    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nxwt\" (UniqueName: \"kubernetes.io/projected/55e5377e-dedf-4cbe-8fed-f6e194f72341-kube-api-access-4nxwt\") pod \"55e5377e-dedf-4cbe-8fed-f6e194f72341\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") "
Jul 21 12:44:18.423088 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.422946    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55e5377e-dedf-4cbe-8fed-f6e194f72341-maas-api-tls\") pod \"55e5377e-dedf-4cbe-8fed-f6e194f72341\" (UID: \"55e5377e-dedf-4cbe-8fed-f6e194f72341\") "
Jul 21 12:44:18.423262 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.423239    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55e5377e-dedf-4cbe-8fed-f6e194f72341-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55e5377e-dedf-4cbe-8fed-f6e194f72341" (UID: "55e5377e-dedf-4cbe-8fed-f6e194f72341"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:44:18.425175 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.425146    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55e5377e-dedf-4cbe-8fed-f6e194f72341-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55e5377e-dedf-4cbe-8fed-f6e194f72341" (UID: "55e5377e-dedf-4cbe-8fed-f6e194f72341"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:44:18.425297 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.425183    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55e5377e-dedf-4cbe-8fed-f6e194f72341-kube-api-access-4nxwt" (OuterVolumeSpecName: "kube-api-access-4nxwt") pod "55e5377e-dedf-4cbe-8fed-f6e194f72341" (UID: "55e5377e-dedf-4cbe-8fed-f6e194f72341"). InnerVolumeSpecName "kube-api-access-4nxwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:44:18.524004 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.523970    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e5377e-dedf-4cbe-8fed-f6e194f72341-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:18.524004 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.523997    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nxwt\" (UniqueName: \"kubernetes.io/projected/55e5377e-dedf-4cbe-8fed-f6e194f72341-kube-api-access-4nxwt\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:18.524004 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:18.524008    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55e5377e-dedf-4cbe-8fed-f6e194f72341-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:19.258646 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:19.258612    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"
Jul 21 12:44:19.258646 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:19.258632    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt" event={"ID":"55e5377e-dedf-4cbe-8fed-f6e194f72341","Type":"ContainerDied","Data":"8e3f40d51963fc3c771853b8e85a3b45326e4dfb313754efae6c42327fd47cc1"}
Jul 21 12:44:19.259307 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:19.258689    2562 scope.go:117] "RemoveContainer" containerID="820772a11508460bb7b910fb79ca6eb944168bdf5070f796d3390ca5e8223b98"
Jul 21 12:44:19.284719 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:19.284695    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"]
Jul 21 12:44:19.288626 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:19.288603    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-21bec010-db9559655-ckdpt"]
Jul 21 12:44:20.416243 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:20.416210    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55e5377e-dedf-4cbe-8fed-f6e194f72341" path="/var/lib/kubelet/pods/55e5377e-dedf-4cbe-8fed-f6e194f72341/volumes"
Jul 21 12:44:21.111243 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:21.111206    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:44:22.830164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:22.830132    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"]
Jul 21 12:44:22.830558 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:22.830338    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy" containerID="cri-o://70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82" gracePeriod=30
Jul 21 12:44:23.707757 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.707717    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"]
Jul 21 12:44:23.708079 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.708065    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy"
Jul 21 12:44:23.708174 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.708080    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy"
Jul 21 12:44:23.708174 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.708097    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55e5377e-dedf-4cbe-8fed-f6e194f72341" containerName="maas-api"
Jul 21 12:44:23.708174 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.708103    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="55e5377e-dedf-4cbe-8fed-f6e194f72341" containerName="maas-api"
Jul 21 12:44:23.708174 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.708148    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="55e5377e-dedf-4cbe-8fed-f6e194f72341" containerName="maas-api"
Jul 21 12:44:23.708174 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.708158    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy"
Jul 21 12:44:23.708174 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.708163    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy"
Jul 21 12:44:23.712937 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.712918    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.723936 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.723891    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-79283f7d-openshift-default-dockercfg-clkp2\""
Jul 21 12:44:23.733164 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.733143    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"]
Jul 21 12:44:23.869920 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.869877    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9d96\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-kube-api-access-f9d96\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.870400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.869958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-socket\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.870400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.870009    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-certs\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.870400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.870065    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9126748-85f4-48cd-a5e4-62282e04165f-istiod-ca-cert\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.870400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.870109    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9126748-85f4-48cd-a5e4-62282e04165f-istio-podinfo\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.870400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.870153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-istio-token\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.870400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.870188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-envoy\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.870400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.870217    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-credential-socket\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.870400 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.870269    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-data\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971058 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.970968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-istio-token\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971058 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-envoy\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971058 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-credential-socket\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971390 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-data\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971390 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971129    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9d96\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-kube-api-access-f9d96\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971390 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-socket\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971390 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-certs\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971390 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9126748-85f4-48cd-a5e4-62282e04165f-istiod-ca-cert\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971656 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9126748-85f4-48cd-a5e4-62282e04165f-istio-podinfo\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971656 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-credential-socket\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.971768 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971683    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-socket\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.972015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971762    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-certs\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.972015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.971829    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-data\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.972128 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.972043    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9126748-85f4-48cd-a5e4-62282e04165f-istiod-ca-cert\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.973587 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.973563    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-envoy\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.973949 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.973928    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9126748-85f4-48cd-a5e4-62282e04165f-istio-podinfo\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.983595 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.983568    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-istio-token\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:23.984514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:23.984492    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9d96\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-kube-api-access-f9d96\") pod \"e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") " pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:24.023071 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:24.023046    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:24.170753 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:24.170727    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"]
Jul 21 12:44:24.172368 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:44:24.172324    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9126748_85f4_48cd_a5e4_62282e04165f.slice/crio-268583615bdb0e7d026e756da782390e3deef9b0f4023afa3adc459ed9fa8c5f WatchSource:0}: Error finding container 268583615bdb0e7d026e756da782390e3deef9b0f4023afa3adc459ed9fa8c5f: Status 404 returned error can't find the container with id 268583615bdb0e7d026e756da782390e3deef9b0f4023afa3adc459ed9fa8c5f
Jul 21 12:44:24.174304 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:24.174271    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:44:24.174385 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:24.174336    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:44:24.174385 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:24.174364    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:44:24.286434 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:24.286385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" event={"ID":"e9126748-85f4-48cd-a5e4-62282e04165f","Type":"ContainerStarted","Data":"12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28"}
Jul 21 12:44:24.286607 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:24.286460    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" event={"ID":"e9126748-85f4-48cd-a5e4-62282e04165f","Type":"ContainerStarted","Data":"268583615bdb0e7d026e756da782390e3deef9b0f4023afa3adc459ed9fa8c5f"}
Jul 21 12:44:24.311217 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:24.311162    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" podStartSLOduration=1.311142378 podStartE2EDuration="1.311142378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:44:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:44:24.308908321 +0000 UTC m=+1988.367339325" watchObservedRunningTime="2026-07-21 12:44:24.311142378 +0000 UTC m=+1988.369573383"
Jul 21 12:44:25.024094 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.024056    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:25.025612 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.025584    2562 patch_prober.go:28] interesting pod/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 21 12:44:25.025731 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.025637    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 21 12:44:25.800072 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.800043    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"]
Jul 21 12:44:25.800434 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.800418    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy"
Jul 21 12:44:25.800487 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.800436    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7919d723-adce-44aa-b544-7582095d7f19" containerName="istio-proxy"
Jul 21 12:44:25.803407 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.803388    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:25.805911 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.805889    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-79283f7d-serving-cert\""
Jul 21 12:44:25.806368 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.806349    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:44:25.807959 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.807941    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:44:25.809695 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.809677    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\""
Jul 21 12:44:25.819681 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.819656    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"]
Jul 21 12:44:25.989667 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.989630    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b458ff-b6f1-4109-b706-1172b07fba0a-maas-api-tls\") pod \"maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:25.989873 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.989676    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssnsj\" (UniqueName: \"kubernetes.io/projected/97b458ff-b6f1-4109-b706-1172b07fba0a-kube-api-access-ssnsj\") pod \"maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:25.989873 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:25.989753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b458ff-b6f1-4109-b706-1172b07fba0a-openshift-service-ca\") pod \"maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:26.027224 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.027196    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:26.090931 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.090901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b458ff-b6f1-4109-b706-1172b07fba0a-openshift-service-ca\") pod \"maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:26.091091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.090971    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b458ff-b6f1-4109-b706-1172b07fba0a-maas-api-tls\") pod \"maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:26.091091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.091000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssnsj\" (UniqueName: \"kubernetes.io/projected/97b458ff-b6f1-4109-b706-1172b07fba0a-kube-api-access-ssnsj\") pod \"maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:26.091516 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.091497    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b458ff-b6f1-4109-b706-1172b07fba0a-openshift-service-ca\") pod \"maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:26.093591 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.093568    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b458ff-b6f1-4109-b706-1172b07fba0a-maas-api-tls\") pod \"maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:26.102402 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.102373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssnsj\" (UniqueName: \"kubernetes.io/projected/97b458ff-b6f1-4109-b706-1172b07fba0a-kube-api-access-ssnsj\") pod \"maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:26.116225 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.116206    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:26.256084 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.256055    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"]
Jul 21 12:44:26.258733 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:44:26.258704    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b458ff_b6f1_4109_b706_1172b07fba0a.slice/crio-117ba2310a140b5abf7b7afb00b14ccbbc31d96b26f96dc2d96184a3bd223f65 WatchSource:0}: Error finding container 117ba2310a140b5abf7b7afb00b14ccbbc31d96b26f96dc2d96184a3bd223f65: Status 404 returned error can't find the container with id 117ba2310a140b5abf7b7afb00b14ccbbc31d96b26f96dc2d96184a3bd223f65
Jul 21 12:44:26.298131 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.298082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574" event={"ID":"97b458ff-b6f1-4109-b706-1172b07fba0a","Type":"ContainerStarted","Data":"117ba2310a140b5abf7b7afb00b14ccbbc31d96b26f96dc2d96184a3bd223f65"}
Jul 21 12:44:26.298343 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.298321    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:26.299350 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:26.299328    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:27.305151 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:27.305104    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574" event={"ID":"97b458ff-b6f1-4109-b706-1172b07fba0a","Type":"ContainerStarted","Data":"dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a"}
Jul 21 12:44:27.305567 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:27.305399    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:27.326443 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:27.326376    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574" podStartSLOduration=1.949052058 podStartE2EDuration="2.326359543s" totalImagesPullingTime="377.307485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:44:27.32538218 +0000 UTC m=+1991.383813181" watchObservedRunningTime="2026-07-21 12:44:27.326359543 +0000 UTC m=+1991.384790547"
Jul 21 12:44:28.077942 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.077916    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:44:28.211328 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211243    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-podinfo\") pod \"79f77eed-5673-49cd-93f3-fce9fd398eb9\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") "
Jul 21 12:44:28.211328 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211291    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-envoy\") pod \"79f77eed-5673-49cd-93f3-fce9fd398eb9\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") "
Jul 21 12:44:28.211328 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211315    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2rwf\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-kube-api-access-h2rwf\") pod \"79f77eed-5673-49cd-93f3-fce9fd398eb9\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") "
Jul 21 12:44:28.211600 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211338    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-certs\") pod \"79f77eed-5673-49cd-93f3-fce9fd398eb9\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") "
Jul 21 12:44:28.211600 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211369    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-token\") pod \"79f77eed-5673-49cd-93f3-fce9fd398eb9\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") "
Jul 21 12:44:28.211600 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211409    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-data\") pod \"79f77eed-5673-49cd-93f3-fce9fd398eb9\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") "
Jul 21 12:44:28.211600 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211438    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f77eed-5673-49cd-93f3-fce9fd398eb9-istiod-ca-cert\") pod \"79f77eed-5673-49cd-93f3-fce9fd398eb9\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") "
Jul 21 12:44:28.211600 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211508    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-credential-socket\") pod \"79f77eed-5673-49cd-93f3-fce9fd398eb9\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") "
Jul 21 12:44:28.211600 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211564    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-socket\") pod \"79f77eed-5673-49cd-93f3-fce9fd398eb9\" (UID: \"79f77eed-5673-49cd-93f3-fce9fd398eb9\") "
Jul 21 12:44:28.211919 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211736    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79f77eed-5673-49cd-93f3-fce9fd398eb9" (UID: "79f77eed-5673-49cd-93f3-fce9fd398eb9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:44:28.211984 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211965    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:28.211984 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.211973    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79f77eed-5673-49cd-93f3-fce9fd398eb9" (UID: "79f77eed-5673-49cd-93f3-fce9fd398eb9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:44:28.212220 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.212187    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79f77eed-5673-49cd-93f3-fce9fd398eb9" (UID: "79f77eed-5673-49cd-93f3-fce9fd398eb9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:44:28.212415 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.212316    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79f77eed-5673-49cd-93f3-fce9fd398eb9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79f77eed-5673-49cd-93f3-fce9fd398eb9" (UID: "79f77eed-5673-49cd-93f3-fce9fd398eb9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:44:28.212609 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.212583    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-data" (OuterVolumeSpecName: "istio-data") pod "79f77eed-5673-49cd-93f3-fce9fd398eb9" (UID: "79f77eed-5673-49cd-93f3-fce9fd398eb9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:44:28.213753 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.213723    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79f77eed-5673-49cd-93f3-fce9fd398eb9" (UID: "79f77eed-5673-49cd-93f3-fce9fd398eb9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:44:28.213882 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.213824    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-token" (OuterVolumeSpecName: "istio-token") pod "79f77eed-5673-49cd-93f3-fce9fd398eb9" (UID: "79f77eed-5673-49cd-93f3-fce9fd398eb9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:44:28.214203 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.214183    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-kube-api-access-h2rwf" (OuterVolumeSpecName: "kube-api-access-h2rwf") pod "79f77eed-5673-49cd-93f3-fce9fd398eb9" (UID: "79f77eed-5673-49cd-93f3-fce9fd398eb9"). InnerVolumeSpecName "kube-api-access-h2rwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:44:28.214280 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.214256    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79f77eed-5673-49cd-93f3-fce9fd398eb9" (UID: "79f77eed-5673-49cd-93f3-fce9fd398eb9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:44:28.311262 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.311225    2562 generic.go:356] "Generic (PLEG): container finished" podID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerID="70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82" exitCode=0
Jul 21 12:44:28.311744 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.311280    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"
Jul 21 12:44:28.311744 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.311328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" event={"ID":"79f77eed-5673-49cd-93f3-fce9fd398eb9","Type":"ContainerDied","Data":"70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82"}
Jul 21 12:44:28.311744 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.311394    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv" event={"ID":"79f77eed-5673-49cd-93f3-fce9fd398eb9","Type":"ContainerDied","Data":"482506c569f305e3699f5105bb85835ea88e75c65dcb074f21af5270b7df8155"}
Jul 21 12:44:28.311744 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.311433    2562 scope.go:117] "RemoveContainer" containerID="70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82"
Jul 21 12:44:28.312344 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.312323    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:28.312464 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.312345    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:28.312464 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.312367    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:28.312464 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.312377    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:28.312464 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.312391    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2rwf\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-kube-api-access-h2rwf\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:28.312464 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.312406    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:28.312464 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.312420    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79f77eed-5673-49cd-93f3-fce9fd398eb9-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:28.312464 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.312431    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79f77eed-5673-49cd-93f3-fce9fd398eb9-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:44:28.320852 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.320836    2562 scope.go:117] "RemoveContainer" containerID="b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8"
Jul 21 12:44:28.329012 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.328987    2562 scope.go:117] "RemoveContainer" containerID="70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82"
Jul 21 12:44:28.329287 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:44:28.329268    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82\": container with ID starting with 70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82 not found: ID does not exist" containerID="70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82"
Jul 21 12:44:28.329338 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.329297    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82"} err="failed to get container status \"70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82\": rpc error: code = NotFound desc = could not find container \"70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82\": container with ID starting with 70fe7e441942d383a3e05775f5bea1805d94cfe37119030ac4eeaa66e6b77d82 not found: ID does not exist"
Jul 21 12:44:28.329338 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.329314    2562 scope.go:117] "RemoveContainer" containerID="b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8"
Jul 21 12:44:28.329562 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:44:28.329546    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8\": container with ID starting with b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8 not found: ID does not exist" containerID="b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8"
Jul 21 12:44:28.329607 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.329566    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8"} err="failed to get container status \"b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8\": rpc error: code = NotFound desc = could not find container \"b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8\": container with ID starting with b24ca71a5291548becce45fbe32792ee9a5db57bd42fe63352398eb53952bef8 not found: ID does not exist"
Jul 21 12:44:28.334452 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.334430    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"]
Jul 21 12:44:28.337162 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.337142    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-21bec010-gw-openshift-default-7b7b9c867f-kltmv"]
Jul 21 12:44:28.417223 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:28.417189    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" path="/var/lib/kubelet/pods/79f77eed-5673-49cd-93f3-fce9fd398eb9/volumes"
Jul 21 12:44:32.200477 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:32.200444    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:44:33.317573 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:33.317545    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:44:40.379513 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:40.379428    2562 generic.go:356] "Generic (PLEG): container finished" podID="e9126748-85f4-48cd-a5e4-62282e04165f" containerID="12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28" exitCode=0
Jul 21 12:44:40.379513 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:40.379492    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" event={"ID":"e9126748-85f4-48cd-a5e4-62282e04165f","Type":"ContainerDied","Data":"12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28"}
Jul 21 12:44:40.380005 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:40.379879    2562 scope.go:117] "RemoveContainer" containerID="12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28"
Jul 21 12:44:40.380909 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:40.380877    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:44:40.381019 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:40.380979    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:44:40.381085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:40.381060    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:44:41.299112 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:41.299071    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:41.385943 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:41.385903    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" event={"ID":"e9126748-85f4-48cd-a5e4-62282e04165f","Type":"ContainerStarted","Data":"0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a"}
Jul 21 12:44:42.023764 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:42.023726    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:42.024879 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:42.024855    2562 patch_prober.go:28] interesting pod/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 21 12:44:42.024981 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:42.024905    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 21 12:44:43.023600 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:43.023560    2562 patch_prober.go:28] interesting pod/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 21 12:44:43.024030 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:43.023624    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 21 12:44:44.024183 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:44.024141    2562 patch_prober.go:28] interesting pod/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 21 12:44:44.024581 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:44.024213    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 21 12:44:45.023402 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:45.023364    2562 patch_prober.go:28] interesting pod/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 21 12:44:45.023580 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:45.023420    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 21 12:44:46.027276 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:46.027246    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:46.027723 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:46.027503    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:44:46.028138 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:44:46.028120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:45:02.167440 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.167401    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"]
Jul 21 12:45:02.167955 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.167621    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574" podUID="97b458ff-b6f1-4109-b706-1172b07fba0a" containerName="maas-api" containerID="cri-o://dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a" gracePeriod=30
Jul 21 12:45:02.414835 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.414808    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:45:02.486692 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.486610    2562 generic.go:356] "Generic (PLEG): container finished" podID="97b458ff-b6f1-4109-b706-1172b07fba0a" containerID="dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a" exitCode=0
Jul 21 12:45:02.486692 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.486655    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"
Jul 21 12:45:02.486888 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.486699    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574" event={"ID":"97b458ff-b6f1-4109-b706-1172b07fba0a","Type":"ContainerDied","Data":"dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a"}
Jul 21 12:45:02.486888 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.486748    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574" event={"ID":"97b458ff-b6f1-4109-b706-1172b07fba0a","Type":"ContainerDied","Data":"117ba2310a140b5abf7b7afb00b14ccbbc31d96b26f96dc2d96184a3bd223f65"}
Jul 21 12:45:02.486888 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.486785    2562 scope.go:117] "RemoveContainer" containerID="dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a"
Jul 21 12:45:02.495566 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.495508    2562 scope.go:117] "RemoveContainer" containerID="dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a"
Jul 21 12:45:02.495792 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:02.495760    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a\": container with ID starting with dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a not found: ID does not exist" containerID="dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a"
Jul 21 12:45:02.495841 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.495806    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a"} err="failed to get container status \"dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a\": rpc error: code = NotFound desc = could not find container \"dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a\": container with ID starting with dc0f19cba07236da087c1e31dcaeec27ba1929504bb5b8876f9657ec6b37fd4a not found: ID does not exist"
Jul 21 12:45:02.497044 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.497027    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b458ff-b6f1-4109-b706-1172b07fba0a-openshift-service-ca\") pod \"97b458ff-b6f1-4109-b706-1172b07fba0a\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") "
Jul 21 12:45:02.497101 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.497057    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssnsj\" (UniqueName: \"kubernetes.io/projected/97b458ff-b6f1-4109-b706-1172b07fba0a-kube-api-access-ssnsj\") pod \"97b458ff-b6f1-4109-b706-1172b07fba0a\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") "
Jul 21 12:45:02.497101 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.497078    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b458ff-b6f1-4109-b706-1172b07fba0a-maas-api-tls\") pod \"97b458ff-b6f1-4109-b706-1172b07fba0a\" (UID: \"97b458ff-b6f1-4109-b706-1172b07fba0a\") "
Jul 21 12:45:02.497386 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.497358    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b458ff-b6f1-4109-b706-1172b07fba0a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "97b458ff-b6f1-4109-b706-1172b07fba0a" (UID: "97b458ff-b6f1-4109-b706-1172b07fba0a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:45:02.497490 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.497439    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b458ff-b6f1-4109-b706-1172b07fba0a-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:02.499199 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.499180    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b458ff-b6f1-4109-b706-1172b07fba0a-kube-api-access-ssnsj" (OuterVolumeSpecName: "kube-api-access-ssnsj") pod "97b458ff-b6f1-4109-b706-1172b07fba0a" (UID: "97b458ff-b6f1-4109-b706-1172b07fba0a"). InnerVolumeSpecName "kube-api-access-ssnsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:45:02.499285 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.499255    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97b458ff-b6f1-4109-b706-1172b07fba0a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "97b458ff-b6f1-4109-b706-1172b07fba0a" (UID: "97b458ff-b6f1-4109-b706-1172b07fba0a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:45:02.598160 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.598113    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssnsj\" (UniqueName: \"kubernetes.io/projected/97b458ff-b6f1-4109-b706-1172b07fba0a-kube-api-access-ssnsj\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:02.598160 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.598155    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b458ff-b6f1-4109-b706-1172b07fba0a-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:02.817811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.817768    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"]
Jul 21 12:45:02.832241 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:02.832209    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-79283f7d-66d9c99dfd-h4574"]
Jul 21 12:45:04.417517 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:04.417482    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97b458ff-b6f1-4109-b706-1172b07fba0a" path="/var/lib/kubelet/pods/97b458ff-b6f1-4109-b706-1172b07fba0a/volumes"
Jul 21 12:45:05.007863 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:05.007824    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:45:07.672159 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:07.672123    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"]
Jul 21 12:45:07.672639 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:07.672424    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy" containerID="cri-o://0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a" gracePeriod=30
Jul 21 12:45:08.434540 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.434505    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"]
Jul 21 12:45:08.434867 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.434849    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy"
Jul 21 12:45:08.434954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.434872    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy"
Jul 21 12:45:08.434954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.434898    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b458ff-b6f1-4109-b706-1172b07fba0a" containerName="maas-api"
Jul 21 12:45:08.434954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.434906    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b458ff-b6f1-4109-b706-1172b07fba0a" containerName="maas-api"
Jul 21 12:45:08.435081 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.435010    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy"
Jul 21 12:45:08.435081 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.435023    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b458ff-b6f1-4109-b706-1172b07fba0a" containerName="maas-api"
Jul 21 12:45:08.435081 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.435036    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy"
Jul 21 12:45:08.438264 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.438245    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.442866 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.442843    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-812e6bee-openshift-default-dockercfg-j85wc\""
Jul 21 12:45:08.473934 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.473900    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"]
Jul 21 12:45:08.547128 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.547087    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-data\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.547280 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.547137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-envoy\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.547280 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.547174    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-certs\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.547280 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.547205    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-podinfo\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.547280 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.547252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4xp2\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-kube-api-access-s4xp2\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.547435 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.547324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-socket\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.547435 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.547357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-credential-socket\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.547435 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.547379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istiod-ca-cert\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.547531 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.547439    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-token\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.648272 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-token\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.648475 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-data\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.648475 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-envoy\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.648475 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648400    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-certs\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.648652 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648580    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-podinfo\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.648705 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4xp2\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-kube-api-access-s4xp2\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.648765 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648730    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-socket\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.648765 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-credential-socket\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.648913 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648761    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-data\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.649012 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.648992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istiod-ca-cert\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.649104 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.649078    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-certs\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.649259 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.649206    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-credential-socket\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.649373 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.649346    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-socket\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.649627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.649608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istiod-ca-cert\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.651341 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.651316    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-envoy\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.651449 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.651431    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-podinfo\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.661942 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.661918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-token\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.666288 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.666266    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4xp2\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-kube-api-access-s4xp2\") pod \"e2e-derive-812e6bee-openshift-default-665765959f-vgmnk\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") " pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.749555 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.749454    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:08.892500 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.892445    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"]
Jul 21 12:45:08.894670 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:45:08.894644    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a0506a7_c1e8_412f_931e_efc4a7eef03b.slice/crio-0a35ecd5f19071672389d03ee4037de0d27db2a85142b3848603c77917b250bd WatchSource:0}: Error finding container 0a35ecd5f19071672389d03ee4037de0d27db2a85142b3848603c77917b250bd: Status 404 returned error can't find the container with id 0a35ecd5f19071672389d03ee4037de0d27db2a85142b3848603c77917b250bd
Jul 21 12:45:08.896762 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.896729    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:45:08.896855 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.896840    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:45:08.896894 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:08.896874    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:45:09.528546 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:09.528502    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" event={"ID":"3a0506a7-c1e8-412f-931e-efc4a7eef03b","Type":"ContainerStarted","Data":"35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d"}
Jul 21 12:45:09.528759 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:09.528565    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" event={"ID":"3a0506a7-c1e8-412f-931e-efc4a7eef03b","Type":"ContainerStarted","Data":"0a35ecd5f19071672389d03ee4037de0d27db2a85142b3848603c77917b250bd"}
Jul 21 12:45:09.556408 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:09.556346    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" podStartSLOduration=1.5563293169999999 podStartE2EDuration="1.556329317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:45:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:45:09.554570496 +0000 UTC m=+2033.613001502" watchObservedRunningTime="2026-07-21 12:45:09.556329317 +0000 UTC m=+2033.614760321"
Jul 21 12:45:09.749651 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:09.749615    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:09.754536 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:09.754507    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:10.533292 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.533253    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:10.534138 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.534119    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:10.982069 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.982034    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"]
Jul 21 12:45:10.982601 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.982576    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy"
Jul 21 12:45:10.982728 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.982610    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f77eed-5673-49cd-93f3-fce9fd398eb9" containerName="istio-proxy"
Jul 21 12:45:10.985891 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.985870    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:10.988188 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.988165    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-812e6bee-serving-cert\""
Jul 21 12:45:10.988308 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.988198    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:45:10.988308 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.988251    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\""
Jul 21 12:45:10.988961 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.988946    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:45:10.994741 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:10.994717    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"]
Jul 21 12:45:11.070826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:11.070757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qrt7\" (UniqueName: \"kubernetes.io/projected/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-kube-api-access-8qrt7\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:11.070826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:11.070820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:11.071034 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:11.070885    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-openshift-service-ca\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:11.172207 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:11.172163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qrt7\" (UniqueName: \"kubernetes.io/projected/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-kube-api-access-8qrt7\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:11.172523 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:11.172492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:11.172670 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:11.172553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-openshift-service-ca\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:11.172670 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:11.172590    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-serving-cert: secret "maas-api-e2e-derive-812e6bee-serving-cert" not found
Jul 21 12:45:11.172837 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:11.172678    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls podName:26152a8e-eb3e-45de-b6a6-e46857ab8e0a nodeName:}" failed. No retries permitted until 2026-07-21 12:45:11.672656815 +0000 UTC m=+2035.731087800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls") pod "maas-api-e2e-derive-812e6bee-d586f8979-6sfr6" (UID: "26152a8e-eb3e-45de-b6a6-e46857ab8e0a") : secret "maas-api-e2e-derive-812e6bee-serving-cert" not found
Jul 21 12:45:11.173211 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:11.173191    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-openshift-service-ca\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:11.184572 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:11.184548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qrt7\" (UniqueName: \"kubernetes.io/projected/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-kube-api-access-8qrt7\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:11.677039 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:11.677007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:11.677222 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:11.677170    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-serving-cert: secret "maas-api-e2e-derive-812e6bee-serving-cert" not found
Jul 21 12:45:11.677291 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:11.677243    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls podName:26152a8e-eb3e-45de-b6a6-e46857ab8e0a nodeName:}" failed. No retries permitted until 2026-07-21 12:45:12.677224187 +0000 UTC m=+2036.735655185 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls") pod "maas-api-e2e-derive-812e6bee-d586f8979-6sfr6" (UID: "26152a8e-eb3e-45de-b6a6-e46857ab8e0a") : secret "maas-api-e2e-derive-812e6bee-serving-cert" not found
Jul 21 12:45:12.687271 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:12.687245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:12.690005 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:12.689983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls\") pod \"maas-api-e2e-derive-812e6bee-d586f8979-6sfr6\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:12.800061 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:12.800032    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:12.945279 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:12.945162    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"]
Jul 21 12:45:12.945279 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:12.945197    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:45:12.947849 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:45:12.947766    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26152a8e_eb3e_45de_b6a6_e46857ab8e0a.slice/crio-91720d53dbd5934c772202b3cda38c2904d51fa3ade6554606b7a414f3446345 WatchSource:0}: Error finding container 91720d53dbd5934c772202b3cda38c2904d51fa3ade6554606b7a414f3446345: Status 404 returned error can't find the container with id 91720d53dbd5934c772202b3cda38c2904d51fa3ade6554606b7a414f3446345
Jul 21 12:45:13.091468 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091435    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9d96\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-kube-api-access-f9d96\") pod \"e9126748-85f4-48cd-a5e4-62282e04165f\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") "
Jul 21 12:45:13.091615 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091479    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-data\") pod \"e9126748-85f4-48cd-a5e4-62282e04165f\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") "
Jul 21 12:45:13.091615 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091522    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-certs\") pod \"e9126748-85f4-48cd-a5e4-62282e04165f\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") "
Jul 21 12:45:13.091615 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091540    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9126748-85f4-48cd-a5e4-62282e04165f-istiod-ca-cert\") pod \"e9126748-85f4-48cd-a5e4-62282e04165f\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") "
Jul 21 12:45:13.091615 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091560    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9126748-85f4-48cd-a5e4-62282e04165f-istio-podinfo\") pod \"e9126748-85f4-48cd-a5e4-62282e04165f\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") "
Jul 21 12:45:13.091615 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091583    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-socket\") pod \"e9126748-85f4-48cd-a5e4-62282e04165f\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") "
Jul 21 12:45:13.091615 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091611    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-credential-socket\") pod \"e9126748-85f4-48cd-a5e4-62282e04165f\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") "
Jul 21 12:45:13.091950 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091658    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-envoy\") pod \"e9126748-85f4-48cd-a5e4-62282e04165f\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") "
Jul 21 12:45:13.091950 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091723    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-istio-token\") pod \"e9126748-85f4-48cd-a5e4-62282e04165f\" (UID: \"e9126748-85f4-48cd-a5e4-62282e04165f\") "
Jul 21 12:45:13.091950 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.091871    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e9126748-85f4-48cd-a5e4-62282e04165f" (UID: "e9126748-85f4-48cd-a5e4-62282e04165f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:13.092103 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.092061    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:13.092161 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.092124    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e9126748-85f4-48cd-a5e4-62282e04165f" (UID: "e9126748-85f4-48cd-a5e4-62282e04165f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:13.092245 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.092222    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e9126748-85f4-48cd-a5e4-62282e04165f" (UID: "e9126748-85f4-48cd-a5e4-62282e04165f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:13.092398 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.092352    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9126748-85f4-48cd-a5e4-62282e04165f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e9126748-85f4-48cd-a5e4-62282e04165f" (UID: "e9126748-85f4-48cd-a5e4-62282e04165f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:45:13.093149 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.093125    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e9126748-85f4-48cd-a5e4-62282e04165f" (UID: "e9126748-85f4-48cd-a5e4-62282e04165f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:13.094104 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.094080    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e9126748-85f4-48cd-a5e4-62282e04165f" (UID: "e9126748-85f4-48cd-a5e4-62282e04165f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:13.094190 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.094143    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9126748-85f4-48cd-a5e4-62282e04165f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e9126748-85f4-48cd-a5e4-62282e04165f" (UID: "e9126748-85f4-48cd-a5e4-62282e04165f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:45:13.094603 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.094585    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-kube-api-access-f9d96" (OuterVolumeSpecName: "kube-api-access-f9d96") pod "e9126748-85f4-48cd-a5e4-62282e04165f" (UID: "e9126748-85f4-48cd-a5e4-62282e04165f"). InnerVolumeSpecName "kube-api-access-f9d96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:45:13.094603 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.094591    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e9126748-85f4-48cd-a5e4-62282e04165f" (UID: "e9126748-85f4-48cd-a5e4-62282e04165f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:45:13.192493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.192410    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:13.192493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.192436    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:13.192493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.192447    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9d96\" (UniqueName: \"kubernetes.io/projected/e9126748-85f4-48cd-a5e4-62282e04165f-kube-api-access-f9d96\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:13.192493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.192456    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:13.192493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.192464    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9126748-85f4-48cd-a5e4-62282e04165f-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:13.192493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.192472    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9126748-85f4-48cd-a5e4-62282e04165f-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:13.192493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.192479    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:13.192493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.192489    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9126748-85f4-48cd-a5e4-62282e04165f-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:13.552107 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.552077    2562 generic.go:356] "Generic (PLEG): container finished" podID="e9126748-85f4-48cd-a5e4-62282e04165f" containerID="0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a" exitCode=0
Jul 21 12:45:13.552242 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.552128    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"
Jul 21 12:45:13.552242 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.552169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" event={"ID":"e9126748-85f4-48cd-a5e4-62282e04165f","Type":"ContainerDied","Data":"0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a"}
Jul 21 12:45:13.552242 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.552224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5" event={"ID":"e9126748-85f4-48cd-a5e4-62282e04165f","Type":"ContainerDied","Data":"268583615bdb0e7d026e756da782390e3deef9b0f4023afa3adc459ed9fa8c5f"}
Jul 21 12:45:13.552391 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.552253    2562 scope.go:117] "RemoveContainer" containerID="0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a"
Jul 21 12:45:13.553932 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.553907    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6" event={"ID":"26152a8e-eb3e-45de-b6a6-e46857ab8e0a","Type":"ContainerStarted","Data":"36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0"}
Jul 21 12:45:13.554086 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.553949    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6" event={"ID":"26152a8e-eb3e-45de-b6a6-e46857ab8e0a","Type":"ContainerStarted","Data":"91720d53dbd5934c772202b3cda38c2904d51fa3ade6554606b7a414f3446345"}
Jul 21 12:45:13.554086 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.554050    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:13.561792 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.561764    2562 scope.go:117] "RemoveContainer" containerID="12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28"
Jul 21 12:45:13.569738 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.569720    2562 scope.go:117] "RemoveContainer" containerID="0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a"
Jul 21 12:45:13.570000 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:13.569982    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a\": container with ID starting with 0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a not found: ID does not exist" containerID="0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a"
Jul 21 12:45:13.570047 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.570010    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a"} err="failed to get container status \"0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a\": rpc error: code = NotFound desc = could not find container \"0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a\": container with ID starting with 0ab10de1c1c7587def9a29727e5629b2aa696908bbcb6a57d121cdf40b6f4b3a not found: ID does not exist"
Jul 21 12:45:13.570047 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.570028    2562 scope.go:117] "RemoveContainer" containerID="12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28"
Jul 21 12:45:13.570282 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:13.570264    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28\": container with ID starting with 12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28 not found: ID does not exist" containerID="12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28"
Jul 21 12:45:13.570323 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.570288    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28"} err="failed to get container status \"12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28\": rpc error: code = NotFound desc = could not find container \"12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28\": container with ID starting with 12433d63a716b416b7c186d55ebf46e5b94b82fe03a2dd8af6be0dcfcb2a2b28 not found: ID does not exist"
Jul 21 12:45:13.579235 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.579189    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6" podStartSLOduration=3.101451111 podStartE2EDuration="3.579173786s" totalImagesPullingTime="477.722675ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:45:13.577412729 +0000 UTC m=+2037.635843725" watchObservedRunningTime="2026-07-21 12:45:13.579173786 +0000 UTC m=+2037.637604791"
Jul 21 12:45:13.592139 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.592111    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"]
Jul 21 12:45:13.595958 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:13.595928    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-79283f7d-openshift-default-5f8795467f-wmnp5"]
Jul 21 12:45:14.416968 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:14.416933    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" path="/var/lib/kubelet/pods/e9126748-85f4-48cd-a5e4-62282e04165f/volumes"
Jul 21 12:45:16.094016 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:16.093972    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:45:19.566893 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:19.566861    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:24.611126 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:24.611093    2562 generic.go:356] "Generic (PLEG): container finished" podID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerID="35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d" exitCode=0
Jul 21 12:45:24.611638 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:24.611170    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" event={"ID":"3a0506a7-c1e8-412f-931e-efc4a7eef03b","Type":"ContainerDied","Data":"35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d"}
Jul 21 12:45:24.611716 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:24.611664    2562 scope.go:117] "RemoveContainer" containerID="35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d"
Jul 21 12:45:24.612699 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:24.612669    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:45:24.612754 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:24.612742    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:45:24.612819 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:24.612798    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:45:25.533621 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:25.533591    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:25.617757 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:25.617727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" event={"ID":"3a0506a7-c1e8-412f-931e-efc4a7eef03b","Type":"ContainerStarted","Data":"4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe"}
Jul 21 12:45:25.751028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:25.750982    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:25.751231 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:25.751120    2562 patch_prober.go:28] interesting pod/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 21 12:45:25.751231 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:25.751165    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 21 12:45:26.750357 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:26.750318    2562 patch_prober.go:28] interesting pod/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 21 12:45:26.751101 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:26.751066    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 21 12:45:27.750464 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:27.750422    2562 patch_prober.go:28] interesting pod/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 21 12:45:27.751038 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:27.750507    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 21 12:45:28.750900 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:28.750860    2562 patch_prober.go:28] interesting pod/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 21 12:45:28.751510 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:28.751475    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 21 12:45:29.764841 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:29.764808    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:29.765288 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:29.765050    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:29.765757 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:29.765734    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:46.260236 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.260152    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"]
Jul 21 12:45:46.260804 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.260417    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6" podUID="26152a8e-eb3e-45de-b6a6-e46857ab8e0a" containerName="maas-api" containerID="cri-o://36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0" gracePeriod=30
Jul 21 12:45:46.512936 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.512872    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:46.574495 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.574454    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls\") pod \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") "
Jul 21 12:45:46.574495 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.574509    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qrt7\" (UniqueName: \"kubernetes.io/projected/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-kube-api-access-8qrt7\") pod \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") "
Jul 21 12:45:46.574705 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.574554    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-openshift-service-ca\") pod \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\" (UID: \"26152a8e-eb3e-45de-b6a6-e46857ab8e0a\") "
Jul 21 12:45:46.575006 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.574973    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26152a8e-eb3e-45de-b6a6-e46857ab8e0a" (UID: "26152a8e-eb3e-45de-b6a6-e46857ab8e0a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:45:46.576655 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.576628    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26152a8e-eb3e-45de-b6a6-e46857ab8e0a" (UID: "26152a8e-eb3e-45de-b6a6-e46857ab8e0a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:45:46.576767 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.576708    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-kube-api-access-8qrt7" (OuterVolumeSpecName: "kube-api-access-8qrt7") pod "26152a8e-eb3e-45de-b6a6-e46857ab8e0a" (UID: "26152a8e-eb3e-45de-b6a6-e46857ab8e0a"). InnerVolumeSpecName "kube-api-access-8qrt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:45:46.675933 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.675886    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:46.675933 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.675927    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:46.675933 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.675936    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qrt7\" (UniqueName: \"kubernetes.io/projected/26152a8e-eb3e-45de-b6a6-e46857ab8e0a-kube-api-access-8qrt7\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:46.719111 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.719076    2562 generic.go:356] "Generic (PLEG): container finished" podID="26152a8e-eb3e-45de-b6a6-e46857ab8e0a" containerID="36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0" exitCode=0
Jul 21 12:45:46.719293 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.719130    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"
Jul 21 12:45:46.719293 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.719159    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6" event={"ID":"26152a8e-eb3e-45de-b6a6-e46857ab8e0a","Type":"ContainerDied","Data":"36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0"}
Jul 21 12:45:46.719293 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.719213    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6" event={"ID":"26152a8e-eb3e-45de-b6a6-e46857ab8e0a","Type":"ContainerDied","Data":"91720d53dbd5934c772202b3cda38c2904d51fa3ade6554606b7a414f3446345"}
Jul 21 12:45:46.719293 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.719235    2562 scope.go:117] "RemoveContainer" containerID="36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0"
Jul 21 12:45:46.734599 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.734578    2562 scope.go:117] "RemoveContainer" containerID="36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0"
Jul 21 12:45:46.734884 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:46.734859    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0\": container with ID starting with 36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0 not found: ID does not exist" containerID="36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0"
Jul 21 12:45:46.734974 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.734897    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0"} err="failed to get container status \"36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0\": rpc error: code = NotFound desc = could not find container \"36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0\": container with ID starting with 36f80dbf83ef77cddc8056492b864a07cbf7b5147a5e5a2bb7d024b82fe3f1d0 not found: ID does not exist"
Jul 21 12:45:46.743977 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.743951    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"]
Jul 21 12:45:46.747357 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:46.747331    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-812e6bee-d586f8979-6sfr6"]
Jul 21 12:45:48.417137 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:48.417101    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26152a8e-eb3e-45de-b6a6-e46857ab8e0a" path="/var/lib/kubelet/pods/26152a8e-eb3e-45de-b6a6-e46857ab8e0a/volumes"
Jul 21 12:45:49.290151 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:49.290109    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:45:51.069944 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:51.069902    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"]
Jul 21 12:45:51.070403 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:51.070152    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy" containerID="cri-o://4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe" gracePeriod=30
Jul 21 12:45:56.309189 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.309164    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:56.359903 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.359817    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-token\") pod \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") "
Jul 21 12:45:56.359903 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.359855    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-certs\") pod \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") "
Jul 21 12:45:56.359903 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.359890    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4xp2\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-kube-api-access-s4xp2\") pod \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") "
Jul 21 12:45:56.360175 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.359922    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-socket\") pod \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") "
Jul 21 12:45:56.360175 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.359957    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-podinfo\") pod \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") "
Jul 21 12:45:56.360175 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.359979    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-credential-socket\") pod \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") "
Jul 21 12:45:56.360175 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360101    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-data\") pod \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") "
Jul 21 12:45:56.360175 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360119    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a0506a7-c1e8-412f-931e-efc4a7eef03b" (UID: "3a0506a7-c1e8-412f-931e-efc4a7eef03b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:56.360436 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360179    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-envoy\") pod \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") "
Jul 21 12:45:56.360436 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360240    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istiod-ca-cert\") pod \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\" (UID: \"3a0506a7-c1e8-412f-931e-efc4a7eef03b\") "
Jul 21 12:45:56.360436 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360282    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a0506a7-c1e8-412f-931e-efc4a7eef03b" (UID: "3a0506a7-c1e8-412f-931e-efc4a7eef03b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:56.360436 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360363    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a0506a7-c1e8-412f-931e-efc4a7eef03b" (UID: "3a0506a7-c1e8-412f-931e-efc4a7eef03b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:56.360649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360560    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:56.360649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360578    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:56.360649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360592    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:56.360841 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.360672    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a0506a7-c1e8-412f-931e-efc4a7eef03b" (UID: "3a0506a7-c1e8-412f-931e-efc4a7eef03b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:45:56.361343 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.361313    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a0506a7-c1e8-412f-931e-efc4a7eef03b" (UID: "3a0506a7-c1e8-412f-931e-efc4a7eef03b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:56.362383 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.362360    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a0506a7-c1e8-412f-931e-efc4a7eef03b" (UID: "3a0506a7-c1e8-412f-931e-efc4a7eef03b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:45:56.362466 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.362440    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a0506a7-c1e8-412f-931e-efc4a7eef03b" (UID: "3a0506a7-c1e8-412f-931e-efc4a7eef03b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:45:56.362594 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.362572    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a0506a7-c1e8-412f-931e-efc4a7eef03b" (UID: "3a0506a7-c1e8-412f-931e-efc4a7eef03b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:45:56.362651 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.362573    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-kube-api-access-s4xp2" (OuterVolumeSpecName: "kube-api-access-s4xp2") pod "3a0506a7-c1e8-412f-931e-efc4a7eef03b" (UID: "3a0506a7-c1e8-412f-931e-efc4a7eef03b"). InnerVolumeSpecName "kube-api-access-s4xp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:45:56.461430 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.461401    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:56.461430 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.461426    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:56.461430 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.461436    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:56.461649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.461444    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:56.461649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.461451    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:56.461649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.461459    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4xp2\" (UniqueName: \"kubernetes.io/projected/3a0506a7-c1e8-412f-931e-efc4a7eef03b-kube-api-access-s4xp2\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:45:56.773479 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.773391    2562 generic.go:356] "Generic (PLEG): container finished" podID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerID="4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe" exitCode=0
Jul 21 12:45:56.773479 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.773446    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"
Jul 21 12:45:56.773479 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.773469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" event={"ID":"3a0506a7-c1e8-412f-931e-efc4a7eef03b","Type":"ContainerDied","Data":"4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe"}
Jul 21 12:45:56.773759 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.773519    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk" event={"ID":"3a0506a7-c1e8-412f-931e-efc4a7eef03b","Type":"ContainerDied","Data":"0a35ecd5f19071672389d03ee4037de0d27db2a85142b3848603c77917b250bd"}
Jul 21 12:45:56.773759 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.773535    2562 scope.go:117] "RemoveContainer" containerID="4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe"
Jul 21 12:45:56.782549 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.782528    2562 scope.go:117] "RemoveContainer" containerID="35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d"
Jul 21 12:45:56.790450 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.790428    2562 scope.go:117] "RemoveContainer" containerID="4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe"
Jul 21 12:45:56.790935 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:56.790906    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe\": container with ID starting with 4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe not found: ID does not exist" containerID="4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe"
Jul 21 12:45:56.791068 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.790943    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe"} err="failed to get container status \"4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe\": rpc error: code = NotFound desc = could not find container \"4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe\": container with ID starting with 4e89026e9d7e7b96b9417b69d15651e895b2ad62b483f19463a4a4d87117eafe not found: ID does not exist"
Jul 21 12:45:56.791068 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.790994    2562 scope.go:117] "RemoveContainer" containerID="35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d"
Jul 21 12:45:56.791400 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:45:56.791374    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d\": container with ID starting with 35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d not found: ID does not exist" containerID="35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d"
Jul 21 12:45:56.791495 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.791408    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d"} err="failed to get container status \"35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d\": rpc error: code = NotFound desc = could not find container \"35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d\": container with ID starting with 35f719622e3fa9b91e03e58812706f56e63bf06753878a43316c269abe74b45d not found: ID does not exist"
Jul 21 12:45:56.793294 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.793275    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"]
Jul 21 12:45:56.796982 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:56.796957    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-812e6bee-openshift-default-665765959f-vgmnk"]
Jul 21 12:45:58.417606 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:45:58.417568    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" path="/var/lib/kubelet/pods/3a0506a7-c1e8-412f-931e-efc4a7eef03b/volumes"
Jul 21 12:47:15.304142 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:15.304059    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:47:24.491814 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:24.491764    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:47:29.010747 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.010713    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"]
Jul 21 12:47:29.011244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011221    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26152a8e-eb3e-45de-b6a6-e46857ab8e0a" containerName="maas-api"
Jul 21 12:47:29.011333 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011249    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="26152a8e-eb3e-45de-b6a6-e46857ab8e0a" containerName="maas-api"
Jul 21 12:47:29.011333 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011276    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy"
Jul 21 12:47:29.011333 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011285    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy"
Jul 21 12:47:29.011333 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011306    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy"
Jul 21 12:47:29.011333 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011316    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy"
Jul 21 12:47:29.011333 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011325    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy"
Jul 21 12:47:29.011333 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011333    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy"
Jul 21 12:47:29.011665 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011410    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy"
Jul 21 12:47:29.011665 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011426    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy"
Jul 21 12:47:29.011665 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011437    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="26152a8e-eb3e-45de-b6a6-e46857ab8e0a" containerName="maas-api"
Jul 21 12:47:29.011665 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.011445    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9126748-85f4-48cd-a5e4-62282e04165f" containerName="istio-proxy"
Jul 21 12:47:29.014513 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.014493    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.016898 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.016878    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cd53929c-openshift-default-dockercfg-65tcn\""
Jul 21 12:47:29.042541 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.042512    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"]
Jul 21 12:47:29.124035 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.123990    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-certs\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.124220 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.124107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-data\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.124220 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.124140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-token\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.124220 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.124156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctfxw\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-kube-api-access-ctfxw\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.124220 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.124177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-socket\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.124371 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.124259    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-envoy\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.124371 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.124358    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istiod-ca-cert\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.124451 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.124408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-credential-socket\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.124485 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.124456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-podinfo\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225455 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-certs\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225672 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225482    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-data\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225672 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-token\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225672 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225615    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctfxw\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-kube-api-access-ctfxw\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225672 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-socket\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225926 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-envoy\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225926 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istiod-ca-cert\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225926 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225820    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-credential-socket\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225926 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225853    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-podinfo\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.225926 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.225919    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-data\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.226262 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.226236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-socket\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.226361 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.226244    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-certs\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.226422 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.226376    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-credential-socket\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.226951 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.226924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istiod-ca-cert\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.228254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.228224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-envoy\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.228363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.228333    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-podinfo\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.234425 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.234402    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-token\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.234964 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.234923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctfxw\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-kube-api-access-ctfxw\") pod \"e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") " pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.324569 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.324540    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:29.469698 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.469666    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"]
Jul 21 12:47:29.472954 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:47:29.472915    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa33e22_1ab2_4ad3_8b8d_27ff8817b270.slice/crio-bae0f15fabf7eb8dd5779e78f43664ea5601f54607b55ca483af7414f55013c9 WatchSource:0}: Error finding container bae0f15fabf7eb8dd5779e78f43664ea5601f54607b55ca483af7414f55013c9: Status 404 returned error can't find the container with id bae0f15fabf7eb8dd5779e78f43664ea5601f54607b55ca483af7414f55013c9
Jul 21 12:47:29.475123 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.475081    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:47:29.475236 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.475147    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:47:29.475236 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:29.475173    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:47:30.210520 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:30.210484    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" event={"ID":"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270","Type":"ContainerStarted","Data":"10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb"}
Jul 21 12:47:30.211033 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:30.210539    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" event={"ID":"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270","Type":"ContainerStarted","Data":"bae0f15fabf7eb8dd5779e78f43664ea5601f54607b55ca483af7414f55013c9"}
Jul 21 12:47:30.234541 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:30.234489    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" podStartSLOduration=2.234475222 podStartE2EDuration="2.234475222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:47:30.23196137 +0000 UTC m=+2174.290392387" watchObservedRunningTime="2026-07-21 12:47:30.234475222 +0000 UTC m=+2174.292906227"
Jul 21 12:47:30.325274 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:30.325236    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:30.330184 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:30.330156    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:31.215139 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:31.215102    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:31.216029 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:31.216011    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:32.509674 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.509637    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"]
Jul 21 12:47:32.510083 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.509953    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy"
Jul 21 12:47:32.510083 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.509964    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy"
Jul 21 12:47:32.510083 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.510042    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a0506a7-c1e8-412f-931e-efc4a7eef03b" containerName="istio-proxy"
Jul 21 12:47:32.513106 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.513082    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.515454 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.515428    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-cd53929c-serving-cert\""
Jul 21 12:47:32.515551 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.515528    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:47:32.515617 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.515578    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:47:32.516564 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.516541    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\""
Jul 21 12:47:32.523240 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.523180    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"]
Jul 21 12:47:32.654895 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.654863    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de75e2f-d4c0-45ea-b88d-3583bf011c26-maas-api-tls\") pod \"maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.655066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.654922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de75e2f-d4c0-45ea-b88d-3583bf011c26-openshift-service-ca\") pod \"maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.655123 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.655051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmdml\" (UniqueName: \"kubernetes.io/projected/2de75e2f-d4c0-45ea-b88d-3583bf011c26-kube-api-access-tmdml\") pod \"maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.756171 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.756131    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmdml\" (UniqueName: \"kubernetes.io/projected/2de75e2f-d4c0-45ea-b88d-3583bf011c26-kube-api-access-tmdml\") pod \"maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.756355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.756206    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de75e2f-d4c0-45ea-b88d-3583bf011c26-maas-api-tls\") pod \"maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.756355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.756264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de75e2f-d4c0-45ea-b88d-3583bf011c26-openshift-service-ca\") pod \"maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.757084 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.757023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de75e2f-d4c0-45ea-b88d-3583bf011c26-openshift-service-ca\") pod \"maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.758745 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.758721    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de75e2f-d4c0-45ea-b88d-3583bf011c26-maas-api-tls\") pod \"maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.766893 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.766844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmdml\" (UniqueName: \"kubernetes.io/projected/2de75e2f-d4c0-45ea-b88d-3583bf011c26-kube-api-access-tmdml\") pod \"maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.827690 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.827663    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:32.960504 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:32.960474    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"]
Jul 21 12:47:32.962432 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:47:32.962401    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de75e2f_d4c0_45ea_b88d_3583bf011c26.slice/crio-a70b6ccab318474ae30bb671745225b0034dcea77163beb3de7b34d6dc12ea5f WatchSource:0}: Error finding container a70b6ccab318474ae30bb671745225b0034dcea77163beb3de7b34d6dc12ea5f: Status 404 returned error can't find the container with id a70b6ccab318474ae30bb671745225b0034dcea77163beb3de7b34d6dc12ea5f
Jul 21 12:47:33.227355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:33.227321    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw" event={"ID":"2de75e2f-d4c0-45ea-b88d-3583bf011c26","Type":"ContainerStarted","Data":"a70b6ccab318474ae30bb671745225b0034dcea77163beb3de7b34d6dc12ea5f"}
Jul 21 12:47:34.233546 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:34.233512    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw" event={"ID":"2de75e2f-d4c0-45ea-b88d-3583bf011c26","Type":"ContainerStarted","Data":"dff57f1895612a1f63ff3c4e33db0816bda6e3c39f17c6f584152160052309f0"}
Jul 21 12:47:34.233962 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:34.233641    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:34.253820 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:34.253735    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw" podStartSLOduration=1.84975119 podStartE2EDuration="2.253714554s" totalImagesPullingTime="403.963364ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:47:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:47:34.252823242 +0000 UTC m=+2178.311254247" watchObservedRunningTime="2026-07-21 12:47:34.253714554 +0000 UTC m=+2178.312145559"
Jul 21 12:47:37.906340 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:37.906304    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:47:40.259152 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:40.259109    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:46.216822 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:46.216758    2562 patch_prober.go:28] interesting pod/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:47:46.217243 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:46.216847    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:47:46.295760 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:46.295727    2562 generic.go:356] "Generic (PLEG): container finished" podID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerID="10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb" exitCode=0
Jul 21 12:47:46.295944 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:46.295804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" event={"ID":"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270","Type":"ContainerDied","Data":"10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb"}
Jul 21 12:47:46.296215 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:46.296201    2562 scope.go:117] "RemoveContainer" containerID="10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb"
Jul 21 12:47:46.297143 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:46.297115    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:47:46.297242 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:46.297179    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:47:46.297242 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:46.297210    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:47:47.302393 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:47.302359    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" event={"ID":"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270","Type":"ContainerStarted","Data":"ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929"}
Jul 21 12:47:47.325737 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:47.325708    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:47.325945 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:47.325924    2562 patch_prober.go:28] interesting pod/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:47:47.325992 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:47.325976    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:47:48.325541 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:48.325505    2562 patch_prober.go:28] interesting pod/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:47:48.325955 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:48.325571    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:47:49.325397 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:49.325360    2562 patch_prober.go:28] interesting pod/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:47:49.325576 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:49.325416    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:47:50.325729 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:50.325693    2562 patch_prober.go:28] interesting pod/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:47:50.326147 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:50.325751    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:47:51.325160 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:51.325121    2562 patch_prober.go:28] interesting pod/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:47:51.325372 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:51.325183    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:47:52.328675 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:52.328646    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:52.329198 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:52.328993    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:52.329749 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:52.329732    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:47:54.209895 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.209859    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"]
Jul 21 12:47:54.210349 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.210112    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw" podUID="2de75e2f-d4c0-45ea-b88d-3583bf011c26" containerName="maas-api" containerID="cri-o://dff57f1895612a1f63ff3c4e33db0816bda6e3c39f17c6f584152160052309f0" gracePeriod=30
Jul 21 12:47:54.337454 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.337420    2562 generic.go:356] "Generic (PLEG): container finished" podID="2de75e2f-d4c0-45ea-b88d-3583bf011c26" containerID="dff57f1895612a1f63ff3c4e33db0816bda6e3c39f17c6f584152160052309f0" exitCode=0
Jul 21 12:47:54.337614 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.337491    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw" event={"ID":"2de75e2f-d4c0-45ea-b88d-3583bf011c26","Type":"ContainerDied","Data":"dff57f1895612a1f63ff3c4e33db0816bda6e3c39f17c6f584152160052309f0"}
Jul 21 12:47:54.455338 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.455307    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:54.519445 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.519367    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de75e2f-d4c0-45ea-b88d-3583bf011c26-openshift-service-ca\") pod \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") "
Jul 21 12:47:54.519445 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.519422    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmdml\" (UniqueName: \"kubernetes.io/projected/2de75e2f-d4c0-45ea-b88d-3583bf011c26-kube-api-access-tmdml\") pod \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") "
Jul 21 12:47:54.519663 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.519453    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de75e2f-d4c0-45ea-b88d-3583bf011c26-maas-api-tls\") pod \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\" (UID: \"2de75e2f-d4c0-45ea-b88d-3583bf011c26\") "
Jul 21 12:47:54.519759 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.519732    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de75e2f-d4c0-45ea-b88d-3583bf011c26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2de75e2f-d4c0-45ea-b88d-3583bf011c26" (UID: "2de75e2f-d4c0-45ea-b88d-3583bf011c26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:47:54.521634 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.521603    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2de75e2f-d4c0-45ea-b88d-3583bf011c26-kube-api-access-tmdml" (OuterVolumeSpecName: "kube-api-access-tmdml") pod "2de75e2f-d4c0-45ea-b88d-3583bf011c26" (UID: "2de75e2f-d4c0-45ea-b88d-3583bf011c26"). InnerVolumeSpecName "kube-api-access-tmdml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:47:54.521745 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.521672    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de75e2f-d4c0-45ea-b88d-3583bf011c26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2de75e2f-d4c0-45ea-b88d-3583bf011c26" (UID: "2de75e2f-d4c0-45ea-b88d-3583bf011c26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:47:54.620669 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.620632    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de75e2f-d4c0-45ea-b88d-3583bf011c26-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:47:54.620669 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.620662    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmdml\" (UniqueName: \"kubernetes.io/projected/2de75e2f-d4c0-45ea-b88d-3583bf011c26-kube-api-access-tmdml\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:47:54.620669 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:54.620672    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de75e2f-d4c0-45ea-b88d-3583bf011c26-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:47:55.343287 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:55.343255    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"
Jul 21 12:47:55.343711 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:55.343259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw" event={"ID":"2de75e2f-d4c0-45ea-b88d-3583bf011c26","Type":"ContainerDied","Data":"a70b6ccab318474ae30bb671745225b0034dcea77163beb3de7b34d6dc12ea5f"}
Jul 21 12:47:55.343711 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:55.343391    2562 scope.go:117] "RemoveContainer" containerID="dff57f1895612a1f63ff3c4e33db0816bda6e3c39f17c6f584152160052309f0"
Jul 21 12:47:55.369313 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:55.369285    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"]
Jul 21 12:47:55.373424 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:55.373398    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd53929c-6656bdbc45-8v8dw"]
Jul 21 12:47:56.417827 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:56.417790    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2de75e2f-d4c0-45ea-b88d-3583bf011c26" path="/var/lib/kubelet/pods/2de75e2f-d4c0-45ea-b88d-3583bf011c26/volumes"
Jul 21 12:47:56.805231 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:47:56.805139    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:48:05.132552 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:05.132514    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"]
Jul 21 12:48:05.133151 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:05.132802    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy" containerID="cri-o://ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929" gracePeriod=30
Jul 21 12:48:07.331287 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:07.331250    2562 patch_prober.go:28] interesting pod/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 12:48:07.331657 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:07.331303    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 12:48:10.376872 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.376847    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:48:10.417858 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.417757    2562 generic.go:356] "Generic (PLEG): container finished" podID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerID="ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929" exitCode=0
Jul 21 12:48:10.417991 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.417868    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"
Jul 21 12:48:10.419851 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.419827    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" event={"ID":"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270","Type":"ContainerDied","Data":"ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929"}
Jul 21 12:48:10.419949 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.419878    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8" event={"ID":"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270","Type":"ContainerDied","Data":"bae0f15fabf7eb8dd5779e78f43664ea5601f54607b55ca483af7414f55013c9"}
Jul 21 12:48:10.419949 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.419904    2562 scope.go:117] "RemoveContainer" containerID="ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929"
Jul 21 12:48:10.428784 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.428748    2562 scope.go:117] "RemoveContainer" containerID="10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb"
Jul 21 12:48:10.436879 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.436863    2562 scope.go:117] "RemoveContainer" containerID="ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929"
Jul 21 12:48:10.437183 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:48:10.437155    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929\": container with ID starting with ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929 not found: ID does not exist" containerID="ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929"
Jul 21 12:48:10.437229 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.437192    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929"} err="failed to get container status \"ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929\": rpc error: code = NotFound desc = could not find container \"ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929\": container with ID starting with ccddb928d0b7417d7f8aa1f64c8457d4b79b737c64542318a732363c61047929 not found: ID does not exist"
Jul 21 12:48:10.437229 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.437210    2562 scope.go:117] "RemoveContainer" containerID="10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb"
Jul 21 12:48:10.437422 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:48:10.437405    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb\": container with ID starting with 10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb not found: ID does not exist" containerID="10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb"
Jul 21 12:48:10.437480 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.437428    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb"} err="failed to get container status \"10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb\": rpc error: code = NotFound desc = could not find container \"10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb\": container with ID starting with 10047f136e810ab964ff092fcae0db736ba36bb86351615841b1bd2a184a96bb not found: ID does not exist"
Jul 21 12:48:10.451711 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.451691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-certs\") pod \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") "
Jul 21 12:48:10.451797 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.451715    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-data\") pod \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") "
Jul 21 12:48:10.451797 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.451731    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-credential-socket\") pod \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") "
Jul 21 12:48:10.451797 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.451760    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-envoy\") pod \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") "
Jul 21 12:48:10.451924 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.451829    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-token\") pod \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") "
Jul 21 12:48:10.451924 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.451856    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-podinfo\") pod \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") "
Jul 21 12:48:10.452012 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.451947    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istiod-ca-cert\") pod \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") "
Jul 21 12:48:10.452012 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.451979    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctfxw\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-kube-api-access-ctfxw\") pod \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") "
Jul 21 12:48:10.452012 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.451993    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" (UID: "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:48:10.452165 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.452009    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-socket\") pod \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\" (UID: \"8fa33e22-1ab2-4ad3-8b8d-27ff8817b270\") "
Jul 21 12:48:10.452165 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.452024    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" (UID: "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:48:10.452295 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.452270    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" (UID: "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:48:10.452295 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.452281    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:48:10.452420 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.452298    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:48:10.452526 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.452502    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" (UID: "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:48:10.452590 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.452566    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-data" (OuterVolumeSpecName: "istio-data") pod "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" (UID: "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:48:10.454511 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.454491    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" (UID: "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:48:10.454604 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.454510    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-kube-api-access-ctfxw" (OuterVolumeSpecName: "kube-api-access-ctfxw") pod "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" (UID: "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270"). InnerVolumeSpecName "kube-api-access-ctfxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:48:10.454656 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.454605    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" (UID: "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:48:10.454656 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.454624    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-token" (OuterVolumeSpecName: "istio-token") pod "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" (UID: "8fa33e22-1ab2-4ad3-8b8d-27ff8817b270"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:48:10.553618 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.553583    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:48:10.553618 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.553610    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:48:10.553618 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.553618    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:48:10.553618 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.553625    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:48:10.553898 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.553633    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:48:10.553898 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.553641    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctfxw\" (UniqueName: \"kubernetes.io/projected/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-kube-api-access-ctfxw\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:48:10.553898 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.553650    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:48:10.741249 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.741215    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"]
Jul 21 12:48:10.750215 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:10.748003    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cd53929c-openshift-default-5d5f65c7f9-j4sc8"]
Jul 21 12:48:12.416726 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:12.416686    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" path="/var/lib/kubelet/pods/8fa33e22-1ab2-4ad3-8b8d-27ff8817b270/volumes"
Jul 21 12:48:25.033965 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.033931    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"]
Jul 21 12:48:25.034363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.034242    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy"
Jul 21 12:48:25.034363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.034255    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy"
Jul 21 12:48:25.034363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.034278    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de75e2f-d4c0-45ea-b88d-3583bf011c26" containerName="maas-api"
Jul 21 12:48:25.034363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.034284    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de75e2f-d4c0-45ea-b88d-3583bf011c26" containerName="maas-api"
Jul 21 12:48:25.034363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.034291    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy"
Jul 21 12:48:25.034363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.034296    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy"
Jul 21 12:48:25.034363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.034350    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de75e2f-d4c0-45ea-b88d-3583bf011c26" containerName="maas-api"
Jul 21 12:48:25.034363 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.034362    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy"
Jul 21 12:48:25.037398 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.037376    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.039901 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.039881    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ffb6e94e-openshift-default-dockercfg-6k6ws\""
Jul 21 12:48:25.048818 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.048792    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"]
Jul 21 12:48:25.175699 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.175663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-podinfo\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.175699 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.175698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-socket\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.175980 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.175718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33ad7478-90f0-409c-83c1-ce2ecedf014e-istiod-ca-cert\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.175980 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.175799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-data\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.175980 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.175893    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-envoy\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.175980 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.175932    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-credential-socket\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.176149 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.175980    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-token\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.176149 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.176003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-certs\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.176149 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.176033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x89v\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-kube-api-access-6x89v\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277064 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277028    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-data\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277080    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-envoy\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-credential-socket\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277125    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-token\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-certs\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x89v\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-kube-api-access-6x89v\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277193    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-podinfo\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-socket\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277254 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33ad7478-90f0-409c-83c1-ce2ecedf014e-istiod-ca-cert\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277642 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277506    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-data\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277642 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-certs\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277722 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-credential-socket\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.277812 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.277763    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-socket\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.278240 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.278176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33ad7478-90f0-409c-83c1-ce2ecedf014e-istiod-ca-cert\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.279751 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.279729    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-envoy\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.279989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.279969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-podinfo\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.285041 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.284983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-token\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.285342 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.285325    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x89v\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-kube-api-access-6x89v\") pod \"e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") " pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.347894 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.347860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:25.487210 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.487185    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"]
Jul 21 12:48:25.489549 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:48:25.489520    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ad7478_90f0_409c_83c1_ce2ecedf014e.slice/crio-036f74d90334eef9533fe8cb91767c4e71e7d1e27fc68343dcdef035b56df5a2 WatchSource:0}: Error finding container 036f74d90334eef9533fe8cb91767c4e71e7d1e27fc68343dcdef035b56df5a2: Status 404 returned error can't find the container with id 036f74d90334eef9533fe8cb91767c4e71e7d1e27fc68343dcdef035b56df5a2
Jul 21 12:48:25.491833 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.491799    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:48:25.491946 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.491904    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:48:25.492014 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:25.491951    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:48:26.492549 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:26.492510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" event={"ID":"33ad7478-90f0-409c-83c1-ce2ecedf014e","Type":"ContainerStarted","Data":"678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96"}
Jul 21 12:48:26.492977 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:26.492574    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" event={"ID":"33ad7478-90f0-409c-83c1-ce2ecedf014e","Type":"ContainerStarted","Data":"036f74d90334eef9533fe8cb91767c4e71e7d1e27fc68343dcdef035b56df5a2"}
Jul 21 12:48:26.513792 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:26.513717    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" podStartSLOduration=1.513701596 podStartE2EDuration="1.513701596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:26.511153764 +0000 UTC m=+2230.569584772" watchObservedRunningTime="2026-07-21 12:48:26.513701596 +0000 UTC m=+2230.572132600"
Jul 21 12:48:27.348459 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:27.348421    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:27.353338 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:27.353307    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:27.497193 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:27.497162    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:27.498107 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:27.498086    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:28.644758 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.644724    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"]
Jul 21 12:48:28.645311 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.645290    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fa33e22-1ab2-4ad3-8b8d-27ff8817b270" containerName="istio-proxy"
Jul 21 12:48:28.648364 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.648347    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.651033 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.651010    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:48:28.651170 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.651007    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ffb6e94e-serving-cert\""
Jul 21 12:48:28.652248 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.652233    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:48:28.652333 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.652296    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\""
Jul 21 12:48:28.662438 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.662416    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"]
Jul 21 12:48:28.805958 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.805922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrgk5\" (UniqueName: \"kubernetes.io/projected/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-kube-api-access-vrgk5\") pod \"maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.806128 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.805978    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-openshift-service-ca\") pod \"maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.806128 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.806046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-maas-api-tls\") pod \"maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.906547 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.906454    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrgk5\" (UniqueName: \"kubernetes.io/projected/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-kube-api-access-vrgk5\") pod \"maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.906547 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.906505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-openshift-service-ca\") pod \"maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.906823 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.906618    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-maas-api-tls\") pod \"maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.907099 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.907076    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-openshift-service-ca\") pod \"maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.909235 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.909214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-maas-api-tls\") pod \"maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.915330 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.915310    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrgk5\" (UniqueName: \"kubernetes.io/projected/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-kube-api-access-vrgk5\") pod \"maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:28.959060 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:28.959020    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:29.101697 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:29.101672    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"]
Jul 21 12:48:29.103828 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:48:29.103800    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa4cfd3_d0b3_491a_913c_446fb18dbc80.slice/crio-7bfa2dc08fa5910d3cf757aaea0fa7411782ad65eedbca0662fbb43104fffbcd WatchSource:0}: Error finding container 7bfa2dc08fa5910d3cf757aaea0fa7411782ad65eedbca0662fbb43104fffbcd: Status 404 returned error can't find the container with id 7bfa2dc08fa5910d3cf757aaea0fa7411782ad65eedbca0662fbb43104fffbcd
Jul 21 12:48:29.105583 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:29.105561    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:48:29.506835 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:29.506803    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx" event={"ID":"aaa4cfd3-d0b3-491a-913c-446fb18dbc80","Type":"ContainerStarted","Data":"7bfa2dc08fa5910d3cf757aaea0fa7411782ad65eedbca0662fbb43104fffbcd"}
Jul 21 12:48:30.513124 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:30.513092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx" event={"ID":"aaa4cfd3-d0b3-491a-913c-446fb18dbc80","Type":"ContainerStarted","Data":"140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667"}
Jul 21 12:48:30.513518 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:30.513205    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:30.534573 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:30.534519    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx" podStartSLOduration=2.137886872 podStartE2EDuration="2.534502836s" totalImagesPullingTime="396.615964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:30.5339253 +0000 UTC m=+2234.592356308" watchObservedRunningTime="2026-07-21 12:48:30.534502836 +0000 UTC m=+2234.592933839"
Jul 21 12:48:34.492673 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:34.492633    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:48:36.524602 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:36.524574    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:48:42.498960 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:42.498870    2562 patch_prober.go:28] interesting pod/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 21 12:48:42.499479 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:42.498963    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 21 12:48:42.581645 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:42.581612    2562 generic.go:356] "Generic (PLEG): container finished" podID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerID="678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96" exitCode=0
Jul 21 12:48:42.581840 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:42.581664    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" event={"ID":"33ad7478-90f0-409c-83c1-ce2ecedf014e","Type":"ContainerDied","Data":"678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96"}
Jul 21 12:48:42.582053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:42.582036    2562 scope.go:117] "RemoveContainer" containerID="678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96"
Jul 21 12:48:42.583086 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:42.583050    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:48:42.583142 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:42.583129    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:48:42.583178 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:42.583161    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:48:43.588443 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:43.588408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" event={"ID":"33ad7478-90f0-409c-83c1-ce2ecedf014e","Type":"ContainerStarted","Data":"ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35"}
Jul 21 12:48:44.348939 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:44.348463    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:44.349251 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:44.348905    2562 patch_prober.go:28] interesting pod/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 21 12:48:44.349431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:44.349398    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 21 12:48:45.348483 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:45.348434    2562 patch_prober.go:28] interesting pod/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 21 12:48:45.349014 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:45.348507    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 21 12:48:46.349522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:46.349070    2562 patch_prober.go:28] interesting pod/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 21 12:48:46.349522 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:46.349136    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 21 12:48:46.712040 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:46.711941    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:48:47.349055 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:47.349016    2562 patch_prober.go:28] interesting pod/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 21 12:48:47.349381 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:47.349349    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 21 12:48:48.353075 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:48.353037    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:48.353549 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:48.353440    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:48.354125 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:48.354107    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:48:53.695643 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:48:53.695605    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:49:15.393172 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:15.393132    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:49:16.746748 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:16.746714    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"]
Jul 21 12:49:16.747216 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:16.746920    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx" podUID="aaa4cfd3-d0b3-491a-913c-446fb18dbc80" containerName="maas-api" containerID="cri-o://140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667" gracePeriod=30
Jul 21 12:49:16.991315 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:16.991283    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:49:17.134435 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.134401    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-openshift-service-ca\") pod \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") "
Jul 21 12:49:17.134435 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.134440    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrgk5\" (UniqueName: \"kubernetes.io/projected/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-kube-api-access-vrgk5\") pod \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") "
Jul 21 12:49:17.134659 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.134481    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-maas-api-tls\") pod \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\" (UID: \"aaa4cfd3-d0b3-491a-913c-446fb18dbc80\") "
Jul 21 12:49:17.134901 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.134861    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aaa4cfd3-d0b3-491a-913c-446fb18dbc80" (UID: "aaa4cfd3-d0b3-491a-913c-446fb18dbc80"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:49:17.136762 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.136741    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-kube-api-access-vrgk5" (OuterVolumeSpecName: "kube-api-access-vrgk5") pod "aaa4cfd3-d0b3-491a-913c-446fb18dbc80" (UID: "aaa4cfd3-d0b3-491a-913c-446fb18dbc80"). InnerVolumeSpecName "kube-api-access-vrgk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:49:17.136863 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.136762    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aaa4cfd3-d0b3-491a-913c-446fb18dbc80" (UID: "aaa4cfd3-d0b3-491a-913c-446fb18dbc80"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:49:17.235863 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.235830    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:49:17.235863 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.235858    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrgk5\" (UniqueName: \"kubernetes.io/projected/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-kube-api-access-vrgk5\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:49:17.235863 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.235867    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa4cfd3-d0b3-491a-913c-446fb18dbc80-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:49:17.753926 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.753885    2562 generic.go:356] "Generic (PLEG): container finished" podID="aaa4cfd3-d0b3-491a-913c-446fb18dbc80" containerID="140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667" exitCode=0
Jul 21 12:49:17.754461 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.753936    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"
Jul 21 12:49:17.754461 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.753939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx" event={"ID":"aaa4cfd3-d0b3-491a-913c-446fb18dbc80","Type":"ContainerDied","Data":"140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667"}
Jul 21 12:49:17.754461 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.754067    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx" event={"ID":"aaa4cfd3-d0b3-491a-913c-446fb18dbc80","Type":"ContainerDied","Data":"7bfa2dc08fa5910d3cf757aaea0fa7411782ad65eedbca0662fbb43104fffbcd"}
Jul 21 12:49:17.754461 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.754095    2562 scope.go:117] "RemoveContainer" containerID="140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667"
Jul 21 12:49:17.762938 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.762921    2562 scope.go:117] "RemoveContainer" containerID="140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667"
Jul 21 12:49:17.763181 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:49:17.763161    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667\": container with ID starting with 140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667 not found: ID does not exist" containerID="140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667"
Jul 21 12:49:17.763241 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.763198    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667"} err="failed to get container status \"140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667\": rpc error: code = NotFound desc = could not find container \"140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667\": container with ID starting with 140d6b2d01ee8f0726978eb14c24f354552d1a990041e159f272b71b8d2ed667 not found: ID does not exist"
Jul 21 12:49:17.778082 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.778053    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"]
Jul 21 12:49:17.782270 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:17.782247    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffb6e94e-7c7c7667b4-p8hvx"]
Jul 21 12:49:18.416819 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:18.416763    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa4cfd3-d0b3-491a-913c-446fb18dbc80" path="/var/lib/kubelet/pods/aaa4cfd3-d0b3-491a-913c-446fb18dbc80/volumes"
Jul 21 12:49:24.395341 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:49:24.395301    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:50:11.543454 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:11.543352    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"]
Jul 21 12:50:11.544048 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:11.543628    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy" containerID="cri-o://ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35" gracePeriod=30
Jul 21 12:50:15.299145 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:15.299107    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:50:16.797310 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.797282    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:50:16.944241 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944212    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6x89v\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-kube-api-access-6x89v\") pod \"33ad7478-90f0-409c-83c1-ce2ecedf014e\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") "
Jul 21 12:50:16.944241 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944255    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-credential-socket\") pod \"33ad7478-90f0-409c-83c1-ce2ecedf014e\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") "
Jul 21 12:50:16.944620 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944299    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-token\") pod \"33ad7478-90f0-409c-83c1-ce2ecedf014e\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") "
Jul 21 12:50:16.944620 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944334    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-certs\") pod \"33ad7478-90f0-409c-83c1-ce2ecedf014e\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") "
Jul 21 12:50:16.944620 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944358    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-podinfo\") pod \"33ad7478-90f0-409c-83c1-ce2ecedf014e\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") "
Jul 21 12:50:16.944620 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944390    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33ad7478-90f0-409c-83c1-ce2ecedf014e-istiod-ca-cert\") pod \"33ad7478-90f0-409c-83c1-ce2ecedf014e\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") "
Jul 21 12:50:16.944620 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944451    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-envoy\") pod \"33ad7478-90f0-409c-83c1-ce2ecedf014e\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") "
Jul 21 12:50:16.944620 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944482    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-socket\") pod \"33ad7478-90f0-409c-83c1-ce2ecedf014e\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") "
Jul 21 12:50:16.944620 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944503    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-data\") pod \"33ad7478-90f0-409c-83c1-ce2ecedf014e\" (UID: \"33ad7478-90f0-409c-83c1-ce2ecedf014e\") "
Jul 21 12:50:16.944620 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944566    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33ad7478-90f0-409c-83c1-ce2ecedf014e" (UID: "33ad7478-90f0-409c-83c1-ce2ecedf014e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:50:16.945079 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944694    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33ad7478-90f0-409c-83c1-ce2ecedf014e" (UID: "33ad7478-90f0-409c-83c1-ce2ecedf014e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:50:16.945079 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944851    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:50:16.945079 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.944871    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:50:16.945244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.945084    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33ad7478-90f0-409c-83c1-ce2ecedf014e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33ad7478-90f0-409c-83c1-ce2ecedf014e" (UID: "33ad7478-90f0-409c-83c1-ce2ecedf014e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:50:16.945414 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.945371    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33ad7478-90f0-409c-83c1-ce2ecedf014e" (UID: "33ad7478-90f0-409c-83c1-ce2ecedf014e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:50:16.945660 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.945521    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-data" (OuterVolumeSpecName: "istio-data") pod "33ad7478-90f0-409c-83c1-ce2ecedf014e" (UID: "33ad7478-90f0-409c-83c1-ce2ecedf014e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:50:16.947342 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.947288    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-kube-api-access-6x89v" (OuterVolumeSpecName: "kube-api-access-6x89v") pod "33ad7478-90f0-409c-83c1-ce2ecedf014e" (UID: "33ad7478-90f0-409c-83c1-ce2ecedf014e"). InnerVolumeSpecName "kube-api-access-6x89v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:50:16.947453 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.947370    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33ad7478-90f0-409c-83c1-ce2ecedf014e" (UID: "33ad7478-90f0-409c-83c1-ce2ecedf014e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:50:16.947453 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.947377    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33ad7478-90f0-409c-83c1-ce2ecedf014e" (UID: "33ad7478-90f0-409c-83c1-ce2ecedf014e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:50:16.947568 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:16.947485    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-token" (OuterVolumeSpecName: "istio-token") pod "33ad7478-90f0-409c-83c1-ce2ecedf014e" (UID: "33ad7478-90f0-409c-83c1-ce2ecedf014e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:50:17.040678 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.040642    2562 generic.go:356] "Generic (PLEG): container finished" podID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerID="ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35" exitCode=0
Jul 21 12:50:17.040860 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.040692    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"
Jul 21 12:50:17.040860 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.040732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" event={"ID":"33ad7478-90f0-409c-83c1-ce2ecedf014e","Type":"ContainerDied","Data":"ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35"}
Jul 21 12:50:17.040860 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.040808    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln" event={"ID":"33ad7478-90f0-409c-83c1-ce2ecedf014e","Type":"ContainerDied","Data":"036f74d90334eef9533fe8cb91767c4e71e7d1e27fc68343dcdef035b56df5a2"}
Jul 21 12:50:17.040860 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.040833    2562 scope.go:117] "RemoveContainer" containerID="ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35"
Jul 21 12:50:17.045224 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.045194    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x89v\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-kube-api-access-6x89v\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:50:17.045224 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.045219    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:50:17.045224 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.045228    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:50:17.045410 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.045235    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33ad7478-90f0-409c-83c1-ce2ecedf014e-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:50:17.045410 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.045244    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:50:17.045410 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.045251    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:50:17.045410 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.045258    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33ad7478-90f0-409c-83c1-ce2ecedf014e-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:50:17.049964 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.049944    2562 scope.go:117] "RemoveContainer" containerID="678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96"
Jul 21 12:50:17.060003 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.059981    2562 scope.go:117] "RemoveContainer" containerID="ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35"
Jul 21 12:50:17.060254 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:50:17.060233    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35\": container with ID starting with ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35 not found: ID does not exist" containerID="ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35"
Jul 21 12:50:17.060334 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.060267    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35"} err="failed to get container status \"ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35\": rpc error: code = NotFound desc = could not find container \"ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35\": container with ID starting with ef57b7f7cc0845e8fafcac69babefb9a12e50f7b65e9d26c2d7c4e19a2b16f35 not found: ID does not exist"
Jul 21 12:50:17.060334 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.060293    2562 scope.go:117] "RemoveContainer" containerID="678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96"
Jul 21 12:50:17.060555 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:50:17.060536    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96\": container with ID starting with 678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96 not found: ID does not exist" containerID="678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96"
Jul 21 12:50:17.060600 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.060561    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96"} err="failed to get container status \"678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96\": rpc error: code = NotFound desc = could not find container \"678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96\": container with ID starting with 678e3742a3461d79a25d0f5e1df6267bf593aa73ca819c6cbc35e2a03c964b96 not found: ID does not exist"
Jul 21 12:50:17.064798 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.064758    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"]
Jul 21 12:50:17.073565 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:17.073543    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ffb6e94e-openshift-default-65f847b57f-stgln"]
Jul 21 12:50:18.417258 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:18.417225    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" path="/var/lib/kubelet/pods/33ad7478-90f0-409c-83c1-ce2ecedf014e/volumes"
Jul 21 12:50:27.493177 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:50:27.493142    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:51:07.177974 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.177941    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"]
Jul 21 12:51:07.178437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.178254    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy"
Jul 21 12:51:07.178437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.178287    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy"
Jul 21 12:51:07.178437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.178298    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy"
Jul 21 12:51:07.178437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.178303    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy"
Jul 21 12:51:07.178437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.178321    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa4cfd3-d0b3-491a-913c-446fb18dbc80" containerName="maas-api"
Jul 21 12:51:07.178437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.178328    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa4cfd3-d0b3-491a-913c-446fb18dbc80" containerName="maas-api"
Jul 21 12:51:07.178437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.178378    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy"
Jul 21 12:51:07.178437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.178385    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa4cfd3-d0b3-491a-913c-446fb18dbc80" containerName="maas-api"
Jul 21 12:51:07.178437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.178409    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ad7478-90f0-409c-83c1-ce2ecedf014e" containerName="istio-proxy"
Jul 21 12:51:07.181348 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.181331    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.183607 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.183580    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1cb55b-openshift-default-dockercfg-cfhfq\""
Jul 21 12:51:07.194714 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.194680    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"]
Jul 21 12:51:07.258838 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.258798    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-envoy\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.258838 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.258843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-socket\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.259099 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.258878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmtfk\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-kube-api-access-rmtfk\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.259099 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.258929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istiod-ca-cert\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.259099 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.258983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-data\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.259099 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.259012    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-podinfo\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.259099 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.259048    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-token\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.259099 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.259086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-credential-socket\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.259425 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.259117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-certs\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.359581 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.359546    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-socket\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.359581 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.359587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtfk\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-kube-api-access-rmtfk\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.359912 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.359607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istiod-ca-cert\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.359912 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.359642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-data\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.359912 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.359676    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-podinfo\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.359912 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.359692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-token\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.359912 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.359716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-credential-socket\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.359912 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.359747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-certs\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.360226 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.360029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-envoy\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.360226 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.360121    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-credential-socket\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.360340 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.360257    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-certs\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.360430 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.360406    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-socket\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.360558 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.360475    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-data\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.360558 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.360497    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istiod-ca-cert\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.362424 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.362395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-podinfo\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.362424 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.362409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-envoy\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.368087 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.368054    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-token\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.368190 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.368111    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmtfk\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-kube-api-access-rmtfk\") pod \"e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") " pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.500651 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.500545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:07.646239 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.646209    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"]
Jul 21 12:51:07.649080 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:51:07.649046    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod586cd4d7_587a_4657_ad6d_cc5ed35beac1.slice/crio-93e405852f1eb0c667c48bfbd72d226a076e63affa915ece2f641383838498e1 WatchSource:0}: Error finding container 93e405852f1eb0c667c48bfbd72d226a076e63affa915ece2f641383838498e1: Status 404 returned error can't find the container with id 93e405852f1eb0c667c48bfbd72d226a076e63affa915ece2f641383838498e1
Jul 21 12:51:07.651722 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.651685    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:07.651817 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.651766    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:07.651867 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:07.651826    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:08.286035 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:08.285992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" event={"ID":"586cd4d7-587a-4657-ad6d-cc5ed35beac1","Type":"ContainerStarted","Data":"eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d"}
Jul 21 12:51:08.286447 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:08.286062    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" event={"ID":"586cd4d7-587a-4657-ad6d-cc5ed35beac1","Type":"ContainerStarted","Data":"93e405852f1eb0c667c48bfbd72d226a076e63affa915ece2f641383838498e1"}
Jul 21 12:51:08.327361 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:08.327309    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" podStartSLOduration=1.327293767 podStartE2EDuration="1.327293767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:08.32685316 +0000 UTC m=+2392.385284164" watchObservedRunningTime="2026-07-21 12:51:08.327293767 +0000 UTC m=+2392.385724772"
Jul 21 12:51:08.501428 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:08.501384    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:08.506321 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:08.506298    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:09.291856 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:09.291814    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:09.293046 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:09.293022    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:10.343911 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.343873    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"]
Jul 21 12:51:10.347244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.347225    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.350963 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.350931    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-1cb55b-serving-cert\""
Jul 21 12:51:10.351109 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.350991    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:51:10.351109 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.351049    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:51:10.352014 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.351995    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\""
Jul 21 12:51:10.359642 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.359618    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"]
Jul 21 12:51:10.490649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.490602    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.490890 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.490704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-maas-api-tls\") pod \"maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.490890 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.490822    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhzb8\" (UniqueName: \"kubernetes.io/projected/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-kube-api-access-jhzb8\") pod \"maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.591322 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.591280    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.591530 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.591332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-maas-api-tls\") pod \"maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.591530 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.591367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhzb8\" (UniqueName: \"kubernetes.io/projected/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-kube-api-access-jhzb8\") pod \"maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.592031 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.592004    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.593997 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.593943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-maas-api-tls\") pod \"maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.599888 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.599866    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhzb8\" (UniqueName: \"kubernetes.io/projected/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-kube-api-access-jhzb8\") pod \"maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.658538 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.658491    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:10.803020 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:10.802990    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"]
Jul 21 12:51:10.804856 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:51:10.804826    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f30fdab_1ad6_4df4_ba37_b131dbdf0296.slice/crio-be241b870d3c302a43460c4cec3013c1cfe4d1e2b36c6ce61495dfd92efbddbf WatchSource:0}: Error finding container be241b870d3c302a43460c4cec3013c1cfe4d1e2b36c6ce61495dfd92efbddbf: Status 404 returned error can't find the container with id be241b870d3c302a43460c4cec3013c1cfe4d1e2b36c6ce61495dfd92efbddbf
Jul 21 12:51:11.303316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:11.303276    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g" event={"ID":"6f30fdab-1ad6-4df4-ba37-b131dbdf0296","Type":"ContainerStarted","Data":"be241b870d3c302a43460c4cec3013c1cfe4d1e2b36c6ce61495dfd92efbddbf"}
Jul 21 12:51:12.309927 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:12.309884    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g" event={"ID":"6f30fdab-1ad6-4df4-ba37-b131dbdf0296","Type":"ContainerStarted","Data":"9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e"}
Jul 21 12:51:12.310478 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:12.310021    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:12.328543 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:12.328495    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g" podStartSLOduration=1.497731913 podStartE2EDuration="2.328479393s" totalImagesPullingTime="830.74748ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:12.326510428 +0000 UTC m=+2396.384941433" watchObservedRunningTime="2026-07-21 12:51:12.328479393 +0000 UTC m=+2396.386910396"
Jul 21 12:51:15.335963 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.335928    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"]
Jul 21 12:51:15.339636 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.339611    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.342010 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.341990    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ded38a-openshift-default-dockercfg-qpxgc\""
Jul 21 12:51:15.352377 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.352344    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"]
Jul 21 12:51:15.434303 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.434260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-credential-socket\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.434514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.434317    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-token\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.434514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.434349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istiod-ca-cert\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.434514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.434482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-data\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.434691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.434538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-certs\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.434691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.434582    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-podinfo\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.434691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.434653    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-socket\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.434900 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.434701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9lpr\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-kube-api-access-c9lpr\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.434900 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.434728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-envoy\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536112 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9lpr\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-kube-api-access-c9lpr\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536112 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-envoy\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536132    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-credential-socket\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-token\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istiod-ca-cert\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-data\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536606 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-credential-socket\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536606 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536559    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-certs\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536718 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536643    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-podinfo\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536718 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536676    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-socket\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.536880 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.536825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-data\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.537035 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.537012    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-socket\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.537115 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.537050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-certs\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.537173 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.537124    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istiod-ca-cert\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.538999 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.538979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-envoy\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.539262 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.539243    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-podinfo\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.544195 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.544169    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-token\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.544431 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.544415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9lpr\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-kube-api-access-c9lpr\") pod \"e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") " pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.651015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.650920    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:15.691375 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.691333    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:51:15.794691 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.794664    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"]
Jul 21 12:51:15.796699 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:51:15.796664    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ebced4_ad13_4eb3_9877_c3a0b5e1de90.slice/crio-fb66300d5420201ca63fb32aac3d9ba5043190e53e3b1a1a33f326622c9d71f2 WatchSource:0}: Error finding container fb66300d5420201ca63fb32aac3d9ba5043190e53e3b1a1a33f326622c9d71f2: Status 404 returned error can't find the container with id fb66300d5420201ca63fb32aac3d9ba5043190e53e3b1a1a33f326622c9d71f2
Jul 21 12:51:15.798795 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.798746    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:15.798857 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.798831    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:15.798903 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:15.798861    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:16.332548 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:16.332503    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" event={"ID":"39ebced4-ad13-4eb3-9877-c3a0b5e1de90","Type":"ContainerStarted","Data":"d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1"}
Jul 21 12:51:16.332733 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:16.332613    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" event={"ID":"39ebced4-ad13-4eb3-9877-c3a0b5e1de90","Type":"ContainerStarted","Data":"fb66300d5420201ca63fb32aac3d9ba5043190e53e3b1a1a33f326622c9d71f2"}
Jul 21 12:51:16.376330 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:16.376227    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" podStartSLOduration=1.376206408 podStartE2EDuration="1.376206408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:16.373101526 +0000 UTC m=+2400.431532531" watchObservedRunningTime="2026-07-21 12:51:16.376206408 +0000 UTC m=+2400.434637414"
Jul 21 12:51:16.651181 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:16.651087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:16.656708 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:16.656671    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:17.337297 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:17.337260    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:17.342575 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:17.342541    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:18.320181 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.320152    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:51:18.518739 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.518704    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"]
Jul 21 12:51:18.527948 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.527896    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:18.530544 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.530519    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ded38a-serving-cert\""
Jul 21 12:51:18.534579 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.534551    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"]
Jul 21 12:51:18.567811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.567757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05b90544-295e-4bb6-9d99-72a2f5e700a8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:18.568167 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.568131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8hjd\" (UniqueName: \"kubernetes.io/projected/05b90544-295e-4bb6-9d99-72a2f5e700a8-kube-api-access-s8hjd\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:18.568365 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.568332    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05b90544-295e-4bb6-9d99-72a2f5e700a8-maas-api-tls\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:18.669523 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.669403    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8hjd\" (UniqueName: \"kubernetes.io/projected/05b90544-295e-4bb6-9d99-72a2f5e700a8-kube-api-access-s8hjd\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:18.669523 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.669466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05b90544-295e-4bb6-9d99-72a2f5e700a8-maas-api-tls\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:18.669811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.669532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05b90544-295e-4bb6-9d99-72a2f5e700a8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:18.669811 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:51:18.669646    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-serving-cert: secret "maas-api-e2e-inf-b-ded38a-serving-cert" not found
Jul 21 12:51:18.669811 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:51:18.669738    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b90544-295e-4bb6-9d99-72a2f5e700a8-maas-api-tls podName:05b90544-295e-4bb6-9d99-72a2f5e700a8 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:19.169715628 +0000 UTC m=+2403.228146612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/05b90544-295e-4bb6-9d99-72a2f5e700a8-maas-api-tls") pod "maas-api-e2e-inf-b-ded38a-647f9958fc-8687k" (UID: "05b90544-295e-4bb6-9d99-72a2f5e700a8") : secret "maas-api-e2e-inf-b-ded38a-serving-cert" not found
Jul 21 12:51:18.670269 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.670246    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05b90544-295e-4bb6-9d99-72a2f5e700a8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:18.683286 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:18.683260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8hjd\" (UniqueName: \"kubernetes.io/projected/05b90544-295e-4bb6-9d99-72a2f5e700a8-kube-api-access-s8hjd\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:19.174034 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:19.173993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05b90544-295e-4bb6-9d99-72a2f5e700a8-maas-api-tls\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:19.176592 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:19.176564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05b90544-295e-4bb6-9d99-72a2f5e700a8-maas-api-tls\") pod \"maas-api-e2e-inf-b-ded38a-647f9958fc-8687k\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:19.453913 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:19.453808    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:19.615102 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:19.615077    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"]
Jul 21 12:51:19.617089 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:51:19.617062    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b90544_295e_4bb6_9d99_72a2f5e700a8.slice/crio-630a8161cb9bff5c459e0fdb3002ee001a7666bbe642d3fc99e45ed1c347ea6a WatchSource:0}: Error finding container 630a8161cb9bff5c459e0fdb3002ee001a7666bbe642d3fc99e45ed1c347ea6a: Status 404 returned error can't find the container with id 630a8161cb9bff5c459e0fdb3002ee001a7666bbe642d3fc99e45ed1c347ea6a
Jul 21 12:51:20.364817 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:20.364759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k" event={"ID":"05b90544-295e-4bb6-9d99-72a2f5e700a8","Type":"ContainerStarted","Data":"fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e"}
Jul 21 12:51:20.365023 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:20.364840    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k" event={"ID":"05b90544-295e-4bb6-9d99-72a2f5e700a8","Type":"ContainerStarted","Data":"630a8161cb9bff5c459e0fdb3002ee001a7666bbe642d3fc99e45ed1c347ea6a"}
Jul 21 12:51:20.365023 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:20.364955    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:20.381155 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:20.381096    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k" podStartSLOduration=2.014484287 podStartE2EDuration="2.381076211s" totalImagesPullingTime="366.591924ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:20.380749522 +0000 UTC m=+2404.439180529" watchObservedRunningTime="2026-07-21 12:51:20.381076211 +0000 UTC m=+2404.439507217"
Jul 21 12:51:23.388584 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:23.388546    2562 generic.go:356] "Generic (PLEG): container finished" podID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerID="eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d" exitCode=0
Jul 21 12:51:23.389006 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:23.388622    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" event={"ID":"586cd4d7-587a-4657-ad6d-cc5ed35beac1","Type":"ContainerDied","Data":"eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d"}
Jul 21 12:51:23.389055 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:23.389005    2562 scope.go:117] "RemoveContainer" containerID="eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d"
Jul 21 12:51:23.389954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:23.389926    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:23.390024 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:23.389990    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:23.390024 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:23.390019    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:24.292422 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:24.292366    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:24.396084 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:24.396043    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" event={"ID":"586cd4d7-587a-4657-ad6d-cc5ed35beac1","Type":"ContainerStarted","Data":"64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7"}
Jul 21 12:51:24.495313 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:24.495279    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:51:24.500851 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:24.500816    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:24.501768 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:24.501746    2562 patch_prober.go:28] interesting pod/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 21 12:51:24.501914 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:24.501816    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 21 12:51:25.501155 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:25.501122    2562 patch_prober.go:28] interesting pod/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 21 12:51:25.501738 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:25.501707    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 21 12:51:26.388372 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:26.388326    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:51:26.501324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:26.501291    2562 patch_prober.go:28] interesting pod/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 21 12:51:26.501693 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:26.501377    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 21 12:51:27.501849 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:27.501799    2562 patch_prober.go:28] interesting pod/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 21 12:51:27.502338 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:27.501882    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 21 12:51:28.505421 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:28.505383    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:28.505817 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:28.505663    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:28.506338 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:28.506316    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:51:32.338940 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:32.338856    2562 patch_prober.go:28] interesting pod/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 21 12:51:32.338940 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:32.338927    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 21 12:51:32.446555 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:32.446463    2562 generic.go:356] "Generic (PLEG): container finished" podID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerID="d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1" exitCode=0
Jul 21 12:51:32.446555 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:32.446516    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" event={"ID":"39ebced4-ad13-4eb3-9877-c3a0b5e1de90","Type":"ContainerDied","Data":"d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1"}
Jul 21 12:51:32.446935 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:32.446921    2562 scope.go:117] "RemoveContainer" containerID="d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1"
Jul 21 12:51:32.447970 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:32.447943    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:32.448034 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:32.448011    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:32.448074 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:32.448041    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:51:33.459014 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:33.458977    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" event={"ID":"39ebced4-ad13-4eb3-9877-c3a0b5e1de90","Type":"ContainerStarted","Data":"ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073"}
Jul 21 12:51:33.652048 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:33.652005    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:33.652530 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:33.652495    2562 patch_prober.go:28] interesting pod/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 21 12:51:33.652664 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:33.652558    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 21 12:51:34.651730 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:34.651681    2562 patch_prober.go:28] interesting pod/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 21 12:51:34.652251 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:34.651768    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 21 12:51:35.651832 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:35.651759    2562 patch_prober.go:28] interesting pod/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 21 12:51:35.652346 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:35.651854    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 21 12:51:36.651883 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:36.651844    2562 patch_prober.go:28] interesting pod/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 21 12:51:36.652508 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:36.652473    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 21 12:51:37.655860 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:37.655769    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:37.656240 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:37.656084    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:37.656816 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:37.656798    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:51:44.089218 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:44.089178    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:51:54.992267 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:51:54.992230    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:52:27.198597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:52:27.198555    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:52:41.901870 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:52:41.901832    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:53:07.496744 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:07.496661    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:53:14.338539 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.338502    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"]
Jul 21 12:53:14.339110 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.338747    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g" podUID="6f30fdab-1ad6-4df4-ba37-b131dbdf0296" containerName="maas-api" containerID="cri-o://9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e" gracePeriod=30
Jul 21 12:53:14.586233 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.586208    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:53:14.700741 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.700638    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-openshift-service-ca\") pod \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") "
Jul 21 12:53:14.700741 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.700687    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-maas-api-tls\") pod \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") "
Jul 21 12:53:14.701005 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.700750    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhzb8\" (UniqueName: \"kubernetes.io/projected/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-kube-api-access-jhzb8\") pod \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\" (UID: \"6f30fdab-1ad6-4df4-ba37-b131dbdf0296\") "
Jul 21 12:53:14.701114 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.701080    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f30fdab-1ad6-4df4-ba37-b131dbdf0296" (UID: "6f30fdab-1ad6-4df4-ba37-b131dbdf0296"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:53:14.702949 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.702926    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-kube-api-access-jhzb8" (OuterVolumeSpecName: "kube-api-access-jhzb8") pod "6f30fdab-1ad6-4df4-ba37-b131dbdf0296" (UID: "6f30fdab-1ad6-4df4-ba37-b131dbdf0296"). InnerVolumeSpecName "kube-api-access-jhzb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:53:14.702949 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.702940    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f30fdab-1ad6-4df4-ba37-b131dbdf0296" (UID: "6f30fdab-1ad6-4df4-ba37-b131dbdf0296"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:53:14.801657 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.801617    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:14.801657 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.801650    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:14.801657 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.801663    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhzb8\" (UniqueName: \"kubernetes.io/projected/6f30fdab-1ad6-4df4-ba37-b131dbdf0296-kube-api-access-jhzb8\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:14.979347 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.979263    2562 generic.go:356] "Generic (PLEG): container finished" podID="6f30fdab-1ad6-4df4-ba37-b131dbdf0296" containerID="9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e" exitCode=0
Jul 21 12:53:14.979347 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.979308    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"
Jul 21 12:53:14.979524 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.979353    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g" event={"ID":"6f30fdab-1ad6-4df4-ba37-b131dbdf0296","Type":"ContainerDied","Data":"9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e"}
Jul 21 12:53:14.979524 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.979408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g" event={"ID":"6f30fdab-1ad6-4df4-ba37-b131dbdf0296","Type":"ContainerDied","Data":"be241b870d3c302a43460c4cec3013c1cfe4d1e2b36c6ce61495dfd92efbddbf"}
Jul 21 12:53:14.979524 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.979433    2562 scope.go:117] "RemoveContainer" containerID="9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e"
Jul 21 12:53:14.989028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.989011    2562 scope.go:117] "RemoveContainer" containerID="9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e"
Jul 21 12:53:14.989279 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:53:14.989262    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e\": container with ID starting with 9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e not found: ID does not exist" containerID="9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e"
Jul 21 12:53:14.989332 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.989286    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e"} err="failed to get container status \"9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e\": rpc error: code = NotFound desc = could not find container \"9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e\": container with ID starting with 9e864bcf0ce63dc9d90393c98421b9ead15ad0d03edcd29903e69843528dc96e not found: ID does not exist"
Jul 21 12:53:15.000021 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:14.999999    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"]
Jul 21 12:53:15.004174 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:15.004154    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cb55b-55948cf957-zr29g"]
Jul 21 12:53:16.417369 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:16.417337    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f30fdab-1ad6-4df4-ba37-b131dbdf0296" path="/var/lib/kubelet/pods/6f30fdab-1ad6-4df4-ba37-b131dbdf0296/volumes"
Jul 21 12:53:20.390886 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:20.390847    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:53:25.797576 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:25.797545    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"]
Jul 21 12:53:25.797984 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:25.797752    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy" containerID="cri-o://64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7" gracePeriod=30
Jul 21 12:53:28.508379 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:28.508341    2562 patch_prober.go:28] interesting pod/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 12:53:28.508763 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:28.508411    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 12:53:31.040864 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.040838    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:53:31.065991 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.065906    2562 generic.go:356] "Generic (PLEG): container finished" podID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerID="64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7" exitCode=0
Jul 21 12:53:31.065991 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.065966    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"
Jul 21 12:53:31.066212 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.065986    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" event={"ID":"586cd4d7-587a-4657-ad6d-cc5ed35beac1","Type":"ContainerDied","Data":"64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7"}
Jul 21 12:53:31.066212 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.066055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv" event={"ID":"586cd4d7-587a-4657-ad6d-cc5ed35beac1","Type":"ContainerDied","Data":"93e405852f1eb0c667c48bfbd72d226a076e63affa915ece2f641383838498e1"}
Jul 21 12:53:31.066212 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.066086    2562 scope.go:117] "RemoveContainer" containerID="64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7"
Jul 21 12:53:31.075253 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.075231    2562 scope.go:117] "RemoveContainer" containerID="eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d"
Jul 21 12:53:31.085154 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.085134    2562 scope.go:117] "RemoveContainer" containerID="64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7"
Jul 21 12:53:31.085435 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:53:31.085414    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7\": container with ID starting with 64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7 not found: ID does not exist" containerID="64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7"
Jul 21 12:53:31.085520 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.085447    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7"} err="failed to get container status \"64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7\": rpc error: code = NotFound desc = could not find container \"64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7\": container with ID starting with 64181f46d1c3047915f8b9c51e9151a59c02e0a0466ae740999d60fe34b31ea7 not found: ID does not exist"
Jul 21 12:53:31.085520 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.085472    2562 scope.go:117] "RemoveContainer" containerID="eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d"
Jul 21 12:53:31.085737 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:53:31.085719    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d\": container with ID starting with eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d not found: ID does not exist" containerID="eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d"
Jul 21 12:53:31.085796 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.085744    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d"} err="failed to get container status \"eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d\": rpc error: code = NotFound desc = could not find container \"eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d\": container with ID starting with eb3a18c9313a917c760d485fed3790f3cbbf6d36db6df7afc8cf9d22b7f9308d not found: ID does not exist"
Jul 21 12:53:31.150870 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.150836    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-token\") pod \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") "
Jul 21 12:53:31.151051 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.150877    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmtfk\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-kube-api-access-rmtfk\") pod \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") "
Jul 21 12:53:31.151051 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.150914    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-socket\") pod \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") "
Jul 21 12:53:31.151051 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.150954    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-certs\") pod \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") "
Jul 21 12:53:31.151051 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151002    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-podinfo\") pod \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") "
Jul 21 12:53:31.151051 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151028    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istiod-ca-cert\") pod \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") "
Jul 21 12:53:31.151305 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151053    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-credential-socket\") pod \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") "
Jul 21 12:53:31.151305 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151089    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-envoy\") pod \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") "
Jul 21 12:53:31.151305 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151120    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-data\") pod \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\" (UID: \"586cd4d7-587a-4657-ad6d-cc5ed35beac1\") "
Jul 21 12:53:31.151469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151304    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "586cd4d7-587a-4657-ad6d-cc5ed35beac1" (UID: "586cd4d7-587a-4657-ad6d-cc5ed35beac1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:31.151469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151316    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "586cd4d7-587a-4657-ad6d-cc5ed35beac1" (UID: "586cd4d7-587a-4657-ad6d-cc5ed35beac1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:31.151469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151402    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:31.151469 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151419    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:31.151697 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151612    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "586cd4d7-587a-4657-ad6d-cc5ed35beac1" (UID: "586cd4d7-587a-4657-ad6d-cc5ed35beac1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:53:31.151697 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.151677    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "586cd4d7-587a-4657-ad6d-cc5ed35beac1" (UID: "586cd4d7-587a-4657-ad6d-cc5ed35beac1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:31.152075 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.152048    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-data" (OuterVolumeSpecName: "istio-data") pod "586cd4d7-587a-4657-ad6d-cc5ed35beac1" (UID: "586cd4d7-587a-4657-ad6d-cc5ed35beac1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:31.153266 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.153246    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-token" (OuterVolumeSpecName: "istio-token") pod "586cd4d7-587a-4657-ad6d-cc5ed35beac1" (UID: "586cd4d7-587a-4657-ad6d-cc5ed35beac1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:53:31.153466 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.153444    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "586cd4d7-587a-4657-ad6d-cc5ed35beac1" (UID: "586cd4d7-587a-4657-ad6d-cc5ed35beac1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:31.153728 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.153709    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "586cd4d7-587a-4657-ad6d-cc5ed35beac1" (UID: "586cd4d7-587a-4657-ad6d-cc5ed35beac1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:53:31.153817 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.153723    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-kube-api-access-rmtfk" (OuterVolumeSpecName: "kube-api-access-rmtfk") pod "586cd4d7-587a-4657-ad6d-cc5ed35beac1" (UID: "586cd4d7-587a-4657-ad6d-cc5ed35beac1"). InnerVolumeSpecName "kube-api-access-rmtfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:53:31.252817 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.252736    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:31.252817 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.252812    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:31.252817 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.252825    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:31.253081 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.252839    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:31.253081 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.252851    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:31.253081 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.252861    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:31.253081 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.252872    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmtfk\" (UniqueName: \"kubernetes.io/projected/586cd4d7-587a-4657-ad6d-cc5ed35beac1-kube-api-access-rmtfk\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:31.389378 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.389350    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"]
Jul 21 12:53:31.393239 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:31.393210    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1cb55b-openshift-default-7779cdf577-j88zv"]
Jul 21 12:53:32.417222 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:32.417186    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" path="/var/lib/kubelet/pods/586cd4d7-587a-4657-ad6d-cc5ed35beac1/volumes"
Jul 21 12:53:36.596202 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:36.596172    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"]
Jul 21 12:53:36.596666 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:36.596380    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k" podUID="05b90544-295e-4bb6-9d99-72a2f5e700a8" containerName="maas-api" containerID="cri-o://fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e" gracePeriod=30
Jul 21 12:53:37.049182 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.049153    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:53:37.102073 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.102036    2562 generic.go:356] "Generic (PLEG): container finished" podID="05b90544-295e-4bb6-9d99-72a2f5e700a8" containerID="fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e" exitCode=0
Jul 21 12:53:37.102277 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.102078    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"
Jul 21 12:53:37.102277 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.102127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k" event={"ID":"05b90544-295e-4bb6-9d99-72a2f5e700a8","Type":"ContainerDied","Data":"fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e"}
Jul 21 12:53:37.102277 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.102217    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k" event={"ID":"05b90544-295e-4bb6-9d99-72a2f5e700a8","Type":"ContainerDied","Data":"630a8161cb9bff5c459e0fdb3002ee001a7666bbe642d3fc99e45ed1c347ea6a"}
Jul 21 12:53:37.102277 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.102237    2562 scope.go:117] "RemoveContainer" containerID="fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e"
Jul 21 12:53:37.110811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.110790    2562 scope.go:117] "RemoveContainer" containerID="fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e"
Jul 21 12:53:37.111057 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:53:37.111030    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e\": container with ID starting with fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e not found: ID does not exist" containerID="fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e"
Jul 21 12:53:37.111152 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.111064    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e"} err="failed to get container status \"fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e\": rpc error: code = NotFound desc = could not find container \"fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e\": container with ID starting with fafeddcc4df444a58c9f2dc55d34315039dba46f6dafc9f91e3c52beee5c588e not found: ID does not exist"
Jul 21 12:53:37.206519 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.206425    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05b90544-295e-4bb6-9d99-72a2f5e700a8-maas-api-tls\") pod \"05b90544-295e-4bb6-9d99-72a2f5e700a8\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") "
Jul 21 12:53:37.206519 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.206490    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8hjd\" (UniqueName: \"kubernetes.io/projected/05b90544-295e-4bb6-9d99-72a2f5e700a8-kube-api-access-s8hjd\") pod \"05b90544-295e-4bb6-9d99-72a2f5e700a8\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") "
Jul 21 12:53:37.206750 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.206522    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05b90544-295e-4bb6-9d99-72a2f5e700a8-openshift-service-ca\") pod \"05b90544-295e-4bb6-9d99-72a2f5e700a8\" (UID: \"05b90544-295e-4bb6-9d99-72a2f5e700a8\") "
Jul 21 12:53:37.206929 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.206907    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05b90544-295e-4bb6-9d99-72a2f5e700a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05b90544-295e-4bb6-9d99-72a2f5e700a8" (UID: "05b90544-295e-4bb6-9d99-72a2f5e700a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:53:37.208820 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.208765    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05b90544-295e-4bb6-9d99-72a2f5e700a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05b90544-295e-4bb6-9d99-72a2f5e700a8" (UID: "05b90544-295e-4bb6-9d99-72a2f5e700a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:53:37.208947 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.208848    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05b90544-295e-4bb6-9d99-72a2f5e700a8-kube-api-access-s8hjd" (OuterVolumeSpecName: "kube-api-access-s8hjd") pod "05b90544-295e-4bb6-9d99-72a2f5e700a8" (UID: "05b90544-295e-4bb6-9d99-72a2f5e700a8"). InnerVolumeSpecName "kube-api-access-s8hjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:53:37.307540 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.307505    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05b90544-295e-4bb6-9d99-72a2f5e700a8-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:37.307540 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.307533    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8hjd\" (UniqueName: \"kubernetes.io/projected/05b90544-295e-4bb6-9d99-72a2f5e700a8-kube-api-access-s8hjd\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:37.307540 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.307545    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05b90544-295e-4bb6-9d99-72a2f5e700a8-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:37.424101 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.424065    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"]
Jul 21 12:53:37.427875 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:37.427842    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ded38a-647f9958fc-8687k"]
Jul 21 12:53:38.417441 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:38.417391    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05b90544-295e-4bb6-9d99-72a2f5e700a8" path="/var/lib/kubelet/pods/05b90544-295e-4bb6-9d99-72a2f5e700a8/volumes"
Jul 21 12:53:39.297420 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:39.297330    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:53:47.778278 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:47.778247    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"]
Jul 21 12:53:47.778734 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:47.778452    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy" containerID="cri-o://ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073" gracePeriod=30
Jul 21 12:53:48.646110 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646059    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"]
Jul 21 12:53:48.646533 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646516    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f30fdab-1ad6-4df4-ba37-b131dbdf0296" containerName="maas-api"
Jul 21 12:53:48.646649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646535    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f30fdab-1ad6-4df4-ba37-b131dbdf0296" containerName="maas-api"
Jul 21 12:53:48.646649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646569    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy"
Jul 21 12:53:48.646649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646578    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy"
Jul 21 12:53:48.646649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646594    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05b90544-295e-4bb6-9d99-72a2f5e700a8" containerName="maas-api"
Jul 21 12:53:48.646649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646602    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="05b90544-295e-4bb6-9d99-72a2f5e700a8" containerName="maas-api"
Jul 21 12:53:48.646853 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646675    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f30fdab-1ad6-4df4-ba37-b131dbdf0296" containerName="maas-api"
Jul 21 12:53:48.646853 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646686    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy"
Jul 21 12:53:48.646853 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646698    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="05b90544-295e-4bb6-9d99-72a2f5e700a8" containerName="maas-api"
Jul 21 12:53:48.646853 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.646708    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy"
Jul 21 12:53:48.650931 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.650906    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.653461 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.653440    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-635000-openshift-default-dockercfg-xjlll\""
Jul 21 12:53:48.664360 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.664336    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"]
Jul 21 12:53:48.712816 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.712756    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-envoy\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.712976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.712868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-credential-socket\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.712976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.712929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9rp9\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-kube-api-access-c9rp9\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.712976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.712950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-socket\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.712976 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.712973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-token\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.713225 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.713002    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-certs\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.713225 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.713031    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23be5521-4120-4dba-afef-150d3ecf1b8d-istiod-ca-cert\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.713225 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.713066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-podinfo\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.713225 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.713110    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-data\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814273 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814230    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-credential-socket\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814736 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814288    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9rp9\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-kube-api-access-c9rp9\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814736 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814318    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-socket\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814736 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-token\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814736 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814370    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-certs\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814736 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23be5521-4120-4dba-afef-150d3ecf1b8d-istiod-ca-cert\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814736 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-podinfo\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814736 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-data\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814736 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814519    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-envoy\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.814736 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814702    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-credential-socket\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.815233 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814912    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-socket\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.815233 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.814959    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-certs\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.815233 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.815052    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-data\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.815233 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.815192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23be5521-4120-4dba-afef-150d3ecf1b8d-istiod-ca-cert\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.817475 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.817445    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-envoy\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.817633 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.817617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-podinfo\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.822447 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.822422    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-token\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.822549 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.822508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9rp9\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-kube-api-access-c9rp9\") pod \"e2e-api-a-635000-openshift-default-569496d79f-wjxzn\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") " pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:48.961512 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:48.961424    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:49.100095 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:49.100060    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"]
Jul 21 12:53:49.101157 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:53:49.101118    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23be5521_4120_4dba_afef_150d3ecf1b8d.slice/crio-8d5977075b05cd0e40063cd4db7e484e3594428834203899f57f22f348e13304 WatchSource:0}: Error finding container 8d5977075b05cd0e40063cd4db7e484e3594428834203899f57f22f348e13304: Status 404 returned error can't find the container with id 8d5977075b05cd0e40063cd4db7e484e3594428834203899f57f22f348e13304
Jul 21 12:53:49.103238 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:49.103205    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:53:49.103330 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:49.103282    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:53:49.103330 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:49.103311    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:53:49.164326 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:49.164300    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" event={"ID":"23be5521-4120-4dba-afef-150d3ecf1b8d","Type":"ContainerStarted","Data":"8d5977075b05cd0e40063cd4db7e484e3594428834203899f57f22f348e13304"}
Jul 21 12:53:50.170029 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:50.169992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" event={"ID":"23be5521-4120-4dba-afef-150d3ecf1b8d","Type":"ContainerStarted","Data":"a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296"}
Jul 21 12:53:50.191517 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:50.191471    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" podStartSLOduration=2.191454925 podStartE2EDuration="2.191454925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:53:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:53:50.190473449 +0000 UTC m=+2554.248904463" watchObservedRunningTime="2026-07-21 12:53:50.191454925 +0000 UTC m=+2554.249885929"
Jul 21 12:53:50.962416 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:50.962376    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:50.967169 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:50.967140    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:51.175048 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:51.175018    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:51.176106 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:51.176090    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:53:52.210616 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.210581    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"]
Jul 21 12:53:52.211097 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.210943    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy"
Jul 21 12:53:52.211097 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.210957    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="586cd4d7-587a-4657-ad6d-cc5ed35beac1" containerName="istio-proxy"
Jul 21 12:53:52.213956 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.213940    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.216281 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.216260    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:53:52.216409 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.216288    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-635000-serving-cert\""
Jul 21 12:53:52.216658 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.216531    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:53:52.217300 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.217283    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\""
Jul 21 12:53:52.223258 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.223236    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"]
Jul 21 12:53:52.345003 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.344972    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-openshift-service-ca\") pod \"maas-api-e2e-api-a-635000-7889b59f44-lq6r6\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.345207 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.345055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmr2f\" (UniqueName: \"kubernetes.io/projected/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-kube-api-access-cmr2f\") pod \"maas-api-e2e-api-a-635000-7889b59f44-lq6r6\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.345207 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.345083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-maas-api-tls\") pod \"maas-api-e2e-api-a-635000-7889b59f44-lq6r6\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.445826 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.445758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmr2f\" (UniqueName: \"kubernetes.io/projected/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-kube-api-access-cmr2f\") pod \"maas-api-e2e-api-a-635000-7889b59f44-lq6r6\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.446047 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.445890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-maas-api-tls\") pod \"maas-api-e2e-api-a-635000-7889b59f44-lq6r6\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.446047 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.445958    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-openshift-service-ca\") pod \"maas-api-e2e-api-a-635000-7889b59f44-lq6r6\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.446658 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.446633    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-openshift-service-ca\") pod \"maas-api-e2e-api-a-635000-7889b59f44-lq6r6\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.448543 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.448516    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-maas-api-tls\") pod \"maas-api-e2e-api-a-635000-7889b59f44-lq6r6\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.465788 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.465716    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmr2f\" (UniqueName: \"kubernetes.io/projected/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-kube-api-access-cmr2f\") pod \"maas-api-e2e-api-a-635000-7889b59f44-lq6r6\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.526643 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.526609    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:52.657941 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.657910    2562 patch_prober.go:28] interesting pod/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 12:53:52.658091 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.657963    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 12:53:52.662799 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.662757    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"]
Jul 21 12:53:52.664808 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:53:52.664764    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3797b4a4_ad6f_480f_ae60_5b0ae3e3a2a4.slice/crio-2e0363641933fddd1a3bc1de7e1c45da68f41977c11235820b508f2b11de6919 WatchSource:0}: Error finding container 2e0363641933fddd1a3bc1de7e1c45da68f41977c11235820b508f2b11de6919: Status 404 returned error can't find the container with id 2e0363641933fddd1a3bc1de7e1c45da68f41977c11235820b508f2b11de6919
Jul 21 12:53:52.666316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:52.666298    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:53:53.018671 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.018647    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:53:53.151055 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151017    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-certs\") pod \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") "
Jul 21 12:53:53.151055 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151056    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-envoy\") pod \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") "
Jul 21 12:53:53.151261 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151073    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-data\") pod \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") "
Jul 21 12:53:53.151261 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151099    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9lpr\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-kube-api-access-c9lpr\") pod \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") "
Jul 21 12:53:53.151261 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151129    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-podinfo\") pod \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") "
Jul 21 12:53:53.151261 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151155    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-credential-socket\") pod \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") "
Jul 21 12:53:53.151261 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151195    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-socket\") pod \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") "
Jul 21 12:53:53.151521 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151287    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-token\") pod \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") "
Jul 21 12:53:53.151521 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151309    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istiod-ca-cert\") pod \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\" (UID: \"39ebced4-ad13-4eb3-9877-c3a0b5e1de90\") "
Jul 21 12:53:53.151521 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151430    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39ebced4-ad13-4eb3-9877-c3a0b5e1de90" (UID: "39ebced4-ad13-4eb3-9877-c3a0b5e1de90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:53.151688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151511    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39ebced4-ad13-4eb3-9877-c3a0b5e1de90" (UID: "39ebced4-ad13-4eb3-9877-c3a0b5e1de90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:53.151688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151630    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:53.151688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151652    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:53.151897 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151758    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39ebced4-ad13-4eb3-9877-c3a0b5e1de90" (UID: "39ebced4-ad13-4eb3-9877-c3a0b5e1de90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:53.152029 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.151976    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39ebced4-ad13-4eb3-9877-c3a0b5e1de90" (UID: "39ebced4-ad13-4eb3-9877-c3a0b5e1de90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:53:53.152318 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.152292    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-data" (OuterVolumeSpecName: "istio-data") pod "39ebced4-ad13-4eb3-9877-c3a0b5e1de90" (UID: "39ebced4-ad13-4eb3-9877-c3a0b5e1de90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:53.153532 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.153499    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-kube-api-access-c9lpr" (OuterVolumeSpecName: "kube-api-access-c9lpr") pod "39ebced4-ad13-4eb3-9877-c3a0b5e1de90" (UID: "39ebced4-ad13-4eb3-9877-c3a0b5e1de90"). InnerVolumeSpecName "kube-api-access-c9lpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:53:53.153532 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.153518    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39ebced4-ad13-4eb3-9877-c3a0b5e1de90" (UID: "39ebced4-ad13-4eb3-9877-c3a0b5e1de90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:53:53.153673 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.153594    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39ebced4-ad13-4eb3-9877-c3a0b5e1de90" (UID: "39ebced4-ad13-4eb3-9877-c3a0b5e1de90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:53:53.153673 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.153639    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-token" (OuterVolumeSpecName: "istio-token") pod "39ebced4-ad13-4eb3-9877-c3a0b5e1de90" (UID: "39ebced4-ad13-4eb3-9877-c3a0b5e1de90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:53:53.186877 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.186844    2562 generic.go:356] "Generic (PLEG): container finished" podID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerID="ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073" exitCode=0
Jul 21 12:53:53.187011 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.186891    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"
Jul 21 12:53:53.187011 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.186924    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" event={"ID":"39ebced4-ad13-4eb3-9877-c3a0b5e1de90","Type":"ContainerDied","Data":"ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073"}
Jul 21 12:53:53.187011 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.186979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd" event={"ID":"39ebced4-ad13-4eb3-9877-c3a0b5e1de90","Type":"ContainerDied","Data":"fb66300d5420201ca63fb32aac3d9ba5043190e53e3b1a1a33f326622c9d71f2"}
Jul 21 12:53:53.187158 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.187011    2562 scope.go:117] "RemoveContainer" containerID="ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073"
Jul 21 12:53:53.188406 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.188325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6" event={"ID":"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4","Type":"ContainerStarted","Data":"2e0363641933fddd1a3bc1de7e1c45da68f41977c11235820b508f2b11de6919"}
Jul 21 12:53:53.196201 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.196185    2562 scope.go:117] "RemoveContainer" containerID="d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1"
Jul 21 12:53:53.204324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.204308    2562 scope.go:117] "RemoveContainer" containerID="ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073"
Jul 21 12:53:53.204585 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:53:53.204568    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073\": container with ID starting with ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073 not found: ID does not exist" containerID="ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073"
Jul 21 12:53:53.204632 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.204591    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073"} err="failed to get container status \"ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073\": rpc error: code = NotFound desc = could not find container \"ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073\": container with ID starting with ec7a4e56f2aaddda40542d1c5b31fb15b10e6691b9e60be11eab8ded7df0f073 not found: ID does not exist"
Jul 21 12:53:53.204632 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.204609    2562 scope.go:117] "RemoveContainer" containerID="d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1"
Jul 21 12:53:53.204841 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:53:53.204820    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1\": container with ID starting with d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1 not found: ID does not exist" containerID="d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1"
Jul 21 12:53:53.204925 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.204846    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1"} err="failed to get container status \"d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1\": rpc error: code = NotFound desc = could not find container \"d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1\": container with ID starting with d7cb61edef18e6905c354756f1fbe1cc09133b17d91323bbc27b2868caa651a1 not found: ID does not exist"
Jul 21 12:53:53.208974 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.208949    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"]
Jul 21 12:53:53.219237 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.216517    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ded38a-openshift-default-56cf4566c7-2wcgd"]
Jul 21 12:53:53.252643 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.252605    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:53.252643 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.252638    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:53.252908 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.252652    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:53.252908 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.252664    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:53.252908 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.252675    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9lpr\" (UniqueName: \"kubernetes.io/projected/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-kube-api-access-c9lpr\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:53.252908 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.252688    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:53.252908 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:53.252701    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ebced4-ad13-4eb3-9877-c3a0b5e1de90-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:53:54.197265 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:54.197225    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6" event={"ID":"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4","Type":"ContainerStarted","Data":"e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509"}
Jul 21 12:53:54.197481 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:54.197387    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:53:54.214967 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:54.214920    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6" podStartSLOduration=1.420219052 podStartE2EDuration="2.214905018s" totalImagesPullingTime="794.685966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:53:54.213744186 +0000 UTC m=+2558.272175190" watchObservedRunningTime="2026-07-21 12:53:54.214905018 +0000 UTC m=+2558.273336021"
Jul 21 12:53:54.418024 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:54.417993    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" path="/var/lib/kubelet/pods/39ebced4-ad13-4eb3-9877-c3a0b5e1de90/volumes"
Jul 21 12:53:57.333306 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.333272    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"]
Jul 21 12:53:57.333756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.333632    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy"
Jul 21 12:53:57.333756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.333645    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy"
Jul 21 12:53:57.333756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.333656    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy"
Jul 21 12:53:57.333756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.333663    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy"
Jul 21 12:53:57.333756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.333730    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy"
Jul 21 12:53:57.333756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.333739    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ebced4-ad13-4eb3-9877-c3a0b5e1de90" containerName="istio-proxy"
Jul 21 12:53:57.337110 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.337090    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.339496 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.339476    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8af997-openshift-default-dockercfg-2jghl\""
Jul 21 12:53:57.351308 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.351277    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"]
Jul 21 12:53:57.489355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.489309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-certs\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.489355 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.489350    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-data\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.489627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.489374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-envoy\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.489627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.489392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-credential-socket\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.489627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.489429    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-socket\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.489627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.489445    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f648e020-c419-4dcb-9ae2-9fda542fee02-istiod-ca-cert\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.489627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.489462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-podinfo\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.489627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.489477    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-token\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.489627 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.489552    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l77v\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-kube-api-access-6l77v\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590082 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.589996    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-certs\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590082 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590034    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-data\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590082 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590056    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-envoy\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-credential-socket\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590217    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-socket\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590243    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f648e020-c419-4dcb-9ae2-9fda542fee02-istiod-ca-cert\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-podinfo\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590287    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-token\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590335    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l77v\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-kube-api-access-6l77v\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590659 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590406    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-data\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590659 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590459    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-certs\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590756 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590707    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-credential-socket\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.590831 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590801    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-socket\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.591011 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.590988    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f648e020-c419-4dcb-9ae2-9fda542fee02-istiod-ca-cert\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.592876 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.592847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-envoy\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.593150 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.593127    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-podinfo\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.598614 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.598576    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-token\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.598849 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.598831    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l77v\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-kube-api-access-6l77v\") pod \"e2e-api-b-8af997-openshift-default-769bdc484f-r7knq\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") " pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.647652 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.647617    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:57.794008 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.793980    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"]
Jul 21 12:53:57.795740 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:53:57.795703    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf648e020_c419_4dcb_9ae2_9fda542fee02.slice/crio-f1f25f9150acffbe859a6d2c27cfebbd321639715def79022cf99c28a45c4e2f WatchSource:0}: Error finding container f1f25f9150acffbe859a6d2c27cfebbd321639715def79022cf99c28a45c4e2f: Status 404 returned error can't find the container with id f1f25f9150acffbe859a6d2c27cfebbd321639715def79022cf99c28a45c4e2f
Jul 21 12:53:57.797996 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.797961    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:53:57.798081 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.798040    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:53:57.798081 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:57.798068    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:53:58.219115 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:58.219027    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" event={"ID":"f648e020-c419-4dcb-9ae2-9fda542fee02","Type":"ContainerStarted","Data":"0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17"}
Jul 21 12:53:58.219115 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:58.219082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" event={"ID":"f648e020-c419-4dcb-9ae2-9fda542fee02","Type":"ContainerStarted","Data":"f1f25f9150acffbe859a6d2c27cfebbd321639715def79022cf99c28a45c4e2f"}
Jul 21 12:53:58.244857 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:58.244733    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" podStartSLOduration=1.244713625 podStartE2EDuration="1.244713625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:53:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:53:58.242479563 +0000 UTC m=+2562.300910558" watchObservedRunningTime="2026-07-21 12:53:58.244713625 +0000 UTC m=+2562.303144629"
Jul 21 12:53:58.648458 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:58.648422    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:58.653270 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:58.653245    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:59.224671 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:59.224641    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:59.225631 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:59.225608    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:53:59.309155 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:53:59.309121    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:54:00.207862 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.207829    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:54:00.365279 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.365246    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"]
Jul 21 12:54:00.369189 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.369163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:00.372067 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.372040    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-8af997-serving-cert\""
Jul 21 12:54:00.385314 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.385284    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"]
Jul 21 12:54:00.519594 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.519503    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ptbc\" (UniqueName: \"kubernetes.io/projected/d7bc63a5-3fa5-43c2-b059-4770e55053b5-kube-api-access-7ptbc\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:00.519594 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.519588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bc63a5-3fa5-43c2-b059-4770e55053b5-openshift-service-ca\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:00.519871 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.519624    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bc63a5-3fa5-43c2-b059-4770e55053b5-maas-api-tls\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:00.620856 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.620811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bc63a5-3fa5-43c2-b059-4770e55053b5-openshift-service-ca\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:00.621055 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.620882    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bc63a5-3fa5-43c2-b059-4770e55053b5-maas-api-tls\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:00.621055 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.620960    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ptbc\" (UniqueName: \"kubernetes.io/projected/d7bc63a5-3fa5-43c2-b059-4770e55053b5-kube-api-access-7ptbc\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:00.621055 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:54:00.621033    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-serving-cert: secret "maas-api-e2e-api-b-8af997-serving-cert" not found
Jul 21 12:54:00.621224 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:54:00.621094    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7bc63a5-3fa5-43c2-b059-4770e55053b5-maas-api-tls podName:d7bc63a5-3fa5-43c2-b059-4770e55053b5 nodeName:}" failed. No retries permitted until 2026-07-21 12:54:01.12107926 +0000 UTC m=+2565.179510242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7bc63a5-3fa5-43c2-b059-4770e55053b5-maas-api-tls") pod "maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr" (UID: "d7bc63a5-3fa5-43c2-b059-4770e55053b5") : secret "maas-api-e2e-api-b-8af997-serving-cert" not found
Jul 21 12:54:00.621471 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.621451    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bc63a5-3fa5-43c2-b059-4770e55053b5-openshift-service-ca\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:00.630154 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:00.630130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ptbc\" (UniqueName: \"kubernetes.io/projected/d7bc63a5-3fa5-43c2-b059-4770e55053b5-kube-api-access-7ptbc\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:01.125683 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:01.125640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bc63a5-3fa5-43c2-b059-4770e55053b5-maas-api-tls\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:01.128459 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:01.128436    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bc63a5-3fa5-43c2-b059-4770e55053b5-maas-api-tls\") pod \"maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:01.282589 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:01.282553    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:01.457799 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:01.457667    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"]
Jul 21 12:54:01.465876 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:54:01.465835    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7bc63a5_3fa5_43c2_b059_4770e55053b5.slice/crio-6ec9a9bb233dc71794358a8e4930493f0dff7f72605a6b23cecdac992e042cb6 WatchSource:0}: Error finding container 6ec9a9bb233dc71794358a8e4930493f0dff7f72605a6b23cecdac992e042cb6: Status 404 returned error can't find the container with id 6ec9a9bb233dc71794358a8e4930493f0dff7f72605a6b23cecdac992e042cb6
Jul 21 12:54:02.257579 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:02.257535    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr" event={"ID":"d7bc63a5-3fa5-43c2-b059-4770e55053b5","Type":"ContainerStarted","Data":"8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a"}
Jul 21 12:54:02.257928 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:02.257905    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr" event={"ID":"d7bc63a5-3fa5-43c2-b059-4770e55053b5","Type":"ContainerStarted","Data":"6ec9a9bb233dc71794358a8e4930493f0dff7f72605a6b23cecdac992e042cb6"}
Jul 21 12:54:02.258811 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:02.258767    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:02.277919 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:02.277868    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr" podStartSLOduration=1.941675823 podStartE2EDuration="2.277854217s" totalImagesPullingTime="336.178394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:54:02.275672412 +0000 UTC m=+2566.334103417" watchObservedRunningTime="2026-07-21 12:54:02.277854217 +0000 UTC m=+2566.336285220"
Jul 21 12:54:07.216922 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:07.216883    2562 patch_prober.go:28] interesting pod/e2e-api-a-635000-openshift-default-569496d79f-wjxzn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 21 12:54:07.217308 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:07.216949    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 21 12:54:08.293144 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:08.293042    2562 generic.go:356] "Generic (PLEG): container finished" podID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerID="a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296" exitCode=0
Jul 21 12:54:08.293144 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:08.293118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" event={"ID":"23be5521-4120-4dba-afef-150d3ecf1b8d","Type":"ContainerDied","Data":"a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296"}
Jul 21 12:54:08.293645 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:08.293498    2562 scope.go:117] "RemoveContainer" containerID="a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296"
Jul 21 12:54:08.294493 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:08.294465    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:54:08.294559 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:08.294530    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:54:08.294596 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:08.294562    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:54:09.279089 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:09.279055    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:09.301128 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:09.301094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" event={"ID":"23be5521-4120-4dba-afef-150d3ecf1b8d","Type":"ContainerStarted","Data":"7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149"}
Jul 21 12:54:09.961887 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:09.961835    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:54:09.962227 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:09.961931    2562 patch_prober.go:28] interesting pod/e2e-api-a-635000-openshift-default-569496d79f-wjxzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 21 12:54:09.962506 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:09.962385    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 21 12:54:10.962344 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:10.962303    2562 patch_prober.go:28] interesting pod/e2e-api-a-635000-openshift-default-569496d79f-wjxzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 21 12:54:10.962977 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:10.962940    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 21 12:54:11.962881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:11.962570    2562 patch_prober.go:28] interesting pod/e2e-api-a-635000-openshift-default-569496d79f-wjxzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 21 12:54:11.962881 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:11.962642    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 21 12:54:12.962752 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:12.962715    2562 patch_prober.go:28] interesting pod/e2e-api-a-635000-openshift-default-569496d79f-wjxzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 21 12:54:12.962979 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:12.962915    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 21 12:54:13.301601 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:13.301469    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:54:13.966733 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:13.966702    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:54:13.967163 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:13.967016    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:54:13.967744 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:13.967725    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:54:14.777432 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:14.776593    2562 patch_prober.go:28] interesting pod/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": read tcp 10.134.0.2:41530->10.134.0.55:15021: read: connection reset by peer" start-of-body=
Jul 21 12:54:14.777432 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:14.776653    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": read tcp 10.134.0.2:41530->10.134.0.55:15021: read: connection reset by peer"
Jul 21 12:54:15.337616 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:15.337580    2562 generic.go:356] "Generic (PLEG): container finished" podID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerID="0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17" exitCode=0
Jul 21 12:54:15.338015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:15.337659    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" event={"ID":"f648e020-c419-4dcb-9ae2-9fda542fee02","Type":"ContainerDied","Data":"0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17"}
Jul 21 12:54:15.338186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:15.338172    2562 scope.go:117] "RemoveContainer" containerID="0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17"
Jul 21 12:54:15.339111 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:15.339082    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:54:15.339184 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:15.339167    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:54:15.339236 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:15.339220    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:54:16.346950 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:16.346907    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" event={"ID":"f648e020-c419-4dcb-9ae2-9fda542fee02","Type":"ContainerStarted","Data":"951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab"}
Jul 21 12:54:16.648764 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:16.648663    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:54:16.648764 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:16.648713    2562 patch_prober.go:28] interesting pod/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 21 12:54:16.649035 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:16.648810    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 21 12:54:17.648275 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:17.648236    2562 patch_prober.go:28] interesting pod/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 21 12:54:17.649294 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:17.648849    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 21 12:54:18.648449 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:18.648402    2562 patch_prober.go:28] interesting pod/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 21 12:54:18.649082 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:18.648478    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 21 12:54:19.648121 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:19.648085    2562 patch_prober.go:28] interesting pod/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 21 12:54:19.648418 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:19.648382    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 21 12:54:20.652169 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:20.652136    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:54:20.652565 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:20.652404    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:54:20.653217 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:20.653197    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:54:28.503446 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.503403    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"]
Jul 21 12:54:28.503939 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.503650    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6" podUID="3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4" containerName="maas-api" containerID="cri-o://e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509" gracePeriod=30
Jul 21 12:54:28.752830 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.752802    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:54:28.778015 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.777935    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmr2f\" (UniqueName: \"kubernetes.io/projected/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-kube-api-access-cmr2f\") pod \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") "
Jul 21 12:54:28.778160 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.778024    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-openshift-service-ca\") pod \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") "
Jul 21 12:54:28.778160 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.778049    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-maas-api-tls\") pod \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\" (UID: \"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4\") "
Jul 21 12:54:28.778315 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.778294    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4" (UID: "3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:54:28.778389 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.778370    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:28.780218 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.780190    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4" (UID: "3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:54:28.780329 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.780253    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-kube-api-access-cmr2f" (OuterVolumeSpecName: "kube-api-access-cmr2f") pod "3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4" (UID: "3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4"). InnerVolumeSpecName "kube-api-access-cmr2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:54:28.879478 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.879440    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmr2f\" (UniqueName: \"kubernetes.io/projected/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-kube-api-access-cmr2f\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:28.879478 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:28.879471    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:29.417730 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:29.417692    2562 generic.go:356] "Generic (PLEG): container finished" podID="3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4" containerID="e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509" exitCode=0
Jul 21 12:54:29.417730 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:29.417727    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"
Jul 21 12:54:29.417996 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:29.417729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6" event={"ID":"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4","Type":"ContainerDied","Data":"e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509"}
Jul 21 12:54:29.417996 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:29.417824    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6" event={"ID":"3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4","Type":"ContainerDied","Data":"2e0363641933fddd1a3bc1de7e1c45da68f41977c11235820b508f2b11de6919"}
Jul 21 12:54:29.417996 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:29.417851    2562 scope.go:117] "RemoveContainer" containerID="e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509"
Jul 21 12:54:29.426910 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:29.426893    2562 scope.go:117] "RemoveContainer" containerID="e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509"
Jul 21 12:54:29.427198 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:54:29.427171    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509\": container with ID starting with e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509 not found: ID does not exist" containerID="e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509"
Jul 21 12:54:29.427248 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:29.427208    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509"} err="failed to get container status \"e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509\": rpc error: code = NotFound desc = could not find container \"e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509\": container with ID starting with e2f3ef1c2801bb5d8f52e453918fdb0454ec9be566a3ff249df91b9c901da509 not found: ID does not exist"
Jul 21 12:54:29.439461 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:29.439435    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"]
Jul 21 12:54:29.442822 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:29.442799    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-635000-7889b59f44-lq6r6"]
Jul 21 12:54:30.416609 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:30.416572    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4" path="/var/lib/kubelet/pods/3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4/volumes"
Jul 21 12:54:30.801144 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:30.801058    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:54:38.986356 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:38.986325    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"]
Jul 21 12:54:38.986786 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:38.986592    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy" containerID="cri-o://7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149" gracePeriod=30
Jul 21 12:54:43.969637 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:43.969599    2562 patch_prober.go:28] interesting pod/e2e-api-a-635000-openshift-default-569496d79f-wjxzn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 12:54:43.970149 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:43.969693    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 12:54:44.241818 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.241792    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:54:44.321219 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321185    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23be5521-4120-4dba-afef-150d3ecf1b8d-istiod-ca-cert\") pod \"23be5521-4120-4dba-afef-150d3ecf1b8d\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") "
Jul 21 12:54:44.321375 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321227    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-socket\") pod \"23be5521-4120-4dba-afef-150d3ecf1b8d\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") "
Jul 21 12:54:44.321375 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321266    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-token\") pod \"23be5521-4120-4dba-afef-150d3ecf1b8d\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") "
Jul 21 12:54:44.321375 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321288    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-certs\") pod \"23be5521-4120-4dba-afef-150d3ecf1b8d\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") "
Jul 21 12:54:44.321375 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321328    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-envoy\") pod \"23be5521-4120-4dba-afef-150d3ecf1b8d\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") "
Jul 21 12:54:44.321375 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321355    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-podinfo\") pod \"23be5521-4120-4dba-afef-150d3ecf1b8d\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") "
Jul 21 12:54:44.321649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321401    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9rp9\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-kube-api-access-c9rp9\") pod \"23be5521-4120-4dba-afef-150d3ecf1b8d\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") "
Jul 21 12:54:44.321649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321435    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-data\") pod \"23be5521-4120-4dba-afef-150d3ecf1b8d\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") "
Jul 21 12:54:44.321649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321469    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-credential-socket\") pod \"23be5521-4120-4dba-afef-150d3ecf1b8d\" (UID: \"23be5521-4120-4dba-afef-150d3ecf1b8d\") "
Jul 21 12:54:44.321649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321557    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23be5521-4120-4dba-afef-150d3ecf1b8d" (UID: "23be5521-4120-4dba-afef-150d3ecf1b8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:54:44.321649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321602    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23be5521-4120-4dba-afef-150d3ecf1b8d" (UID: "23be5521-4120-4dba-afef-150d3ecf1b8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:54:44.321649 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321610    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23be5521-4120-4dba-afef-150d3ecf1b8d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23be5521-4120-4dba-afef-150d3ecf1b8d" (UID: "23be5521-4120-4dba-afef-150d3ecf1b8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:54:44.321991 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321806    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23be5521-4120-4dba-afef-150d3ecf1b8d" (UID: "23be5521-4120-4dba-afef-150d3ecf1b8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:54:44.321991 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321872    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23be5521-4120-4dba-afef-150d3ecf1b8d-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:44.321991 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321891    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:44.321991 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.321906    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:44.322797 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.322746    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-data" (OuterVolumeSpecName: "istio-data") pod "23be5521-4120-4dba-afef-150d3ecf1b8d" (UID: "23be5521-4120-4dba-afef-150d3ecf1b8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:54:44.323850 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.323814    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-token" (OuterVolumeSpecName: "istio-token") pod "23be5521-4120-4dba-afef-150d3ecf1b8d" (UID: "23be5521-4120-4dba-afef-150d3ecf1b8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:54:44.323850 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.323827    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23be5521-4120-4dba-afef-150d3ecf1b8d" (UID: "23be5521-4120-4dba-afef-150d3ecf1b8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:54:44.323990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.323899    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-kube-api-access-c9rp9" (OuterVolumeSpecName: "kube-api-access-c9rp9") pod "23be5521-4120-4dba-afef-150d3ecf1b8d" (UID: "23be5521-4120-4dba-afef-150d3ecf1b8d"). InnerVolumeSpecName "kube-api-access-c9rp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:54:44.323990 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.323944    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23be5521-4120-4dba-afef-150d3ecf1b8d" (UID: "23be5521-4120-4dba-afef-150d3ecf1b8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:54:44.422434 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.422403    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:44.422434 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.422428    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:44.422434 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.422438    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:44.422651 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.422446    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9rp9\" (UniqueName: \"kubernetes.io/projected/23be5521-4120-4dba-afef-150d3ecf1b8d-kube-api-access-c9rp9\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:44.422651 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.422457    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:44.422651 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.422465    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23be5521-4120-4dba-afef-150d3ecf1b8d-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:44.490500 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.490413    2562 generic.go:356] "Generic (PLEG): container finished" podID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerID="7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149" exitCode=0
Jul 21 12:54:44.490500 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.490469    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"
Jul 21 12:54:44.490696 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.490498    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" event={"ID":"23be5521-4120-4dba-afef-150d3ecf1b8d","Type":"ContainerDied","Data":"7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149"}
Jul 21 12:54:44.490696 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.490551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn" event={"ID":"23be5521-4120-4dba-afef-150d3ecf1b8d","Type":"ContainerDied","Data":"8d5977075b05cd0e40063cd4db7e484e3594428834203899f57f22f348e13304"}
Jul 21 12:54:44.490696 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.490573    2562 scope.go:117] "RemoveContainer" containerID="7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149"
Jul 21 12:54:44.499234 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.499221    2562 scope.go:117] "RemoveContainer" containerID="a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296"
Jul 21 12:54:44.507066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.507049    2562 scope.go:117] "RemoveContainer" containerID="7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149"
Jul 21 12:54:44.507298 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:54:44.507282    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149\": container with ID starting with 7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149 not found: ID does not exist" containerID="7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149"
Jul 21 12:54:44.507343 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.507305    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149"} err="failed to get container status \"7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149\": rpc error: code = NotFound desc = could not find container \"7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149\": container with ID starting with 7568ac2befd520ba7b8cf455660919c1be65b3b9b6277ee40b3db0013757c149 not found: ID does not exist"
Jul 21 12:54:44.507343 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.507320    2562 scope.go:117] "RemoveContainer" containerID="a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296"
Jul 21 12:54:44.507543 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:54:44.507529    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296\": container with ID starting with a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296 not found: ID does not exist" containerID="a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296"
Jul 21 12:54:44.507586 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.507545    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296"} err="failed to get container status \"a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296\": rpc error: code = NotFound desc = could not find container \"a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296\": container with ID starting with a49479e9765b2ac49cf28731ee93fcecfd3505273db7b44e1e419159c58f7296 not found: ID does not exist"
Jul 21 12:54:44.527911 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.527880    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"]
Jul 21 12:54:44.534338 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:44.534315    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-635000-openshift-default-569496d79f-wjxzn"]
Jul 21 12:54:46.416924 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:46.416887    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" path="/var/lib/kubelet/pods/23be5521-4120-4dba-afef-150d3ecf1b8d/volumes"
Jul 21 12:54:55.025526 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.025492    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"]
Jul 21 12:54:55.026053 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.025707    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr" podUID="d7bc63a5-3fa5-43c2-b059-4770e55053b5" containerName="maas-api" containerID="cri-o://8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a" gracePeriod=30
Jul 21 12:54:55.273144 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.273119    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:55.416085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.416049    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ptbc\" (UniqueName: \"kubernetes.io/projected/d7bc63a5-3fa5-43c2-b059-4770e55053b5-kube-api-access-7ptbc\") pod \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") "
Jul 21 12:54:55.416265 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.416115    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bc63a5-3fa5-43c2-b059-4770e55053b5-maas-api-tls\") pod \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") "
Jul 21 12:54:55.416316 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.416287    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bc63a5-3fa5-43c2-b059-4770e55053b5-openshift-service-ca\") pod \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\" (UID: \"d7bc63a5-3fa5-43c2-b059-4770e55053b5\") "
Jul 21 12:54:55.416626 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.416599    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7bc63a5-3fa5-43c2-b059-4770e55053b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7bc63a5-3fa5-43c2-b059-4770e55053b5" (UID: "d7bc63a5-3fa5-43c2-b059-4770e55053b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:54:55.418352 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.418330    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bc63a5-3fa5-43c2-b059-4770e55053b5-kube-api-access-7ptbc" (OuterVolumeSpecName: "kube-api-access-7ptbc") pod "d7bc63a5-3fa5-43c2-b059-4770e55053b5" (UID: "d7bc63a5-3fa5-43c2-b059-4770e55053b5"). InnerVolumeSpecName "kube-api-access-7ptbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:54:55.418443 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.418332    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bc63a5-3fa5-43c2-b059-4770e55053b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7bc63a5-3fa5-43c2-b059-4770e55053b5" (UID: "d7bc63a5-3fa5-43c2-b059-4770e55053b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:54:55.517946 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.517898    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7bc63a5-3fa5-43c2-b059-4770e55053b5-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:55.517946 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.517940    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ptbc\" (UniqueName: \"kubernetes.io/projected/d7bc63a5-3fa5-43c2-b059-4770e55053b5-kube-api-access-7ptbc\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:55.517946 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.517950    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7bc63a5-3fa5-43c2-b059-4770e55053b5-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:54:55.551858 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.551820    2562 generic.go:356] "Generic (PLEG): container finished" podID="d7bc63a5-3fa5-43c2-b059-4770e55053b5" containerID="8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a" exitCode=0
Jul 21 12:54:55.552028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.551855    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr" event={"ID":"d7bc63a5-3fa5-43c2-b059-4770e55053b5","Type":"ContainerDied","Data":"8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a"}
Jul 21 12:54:55.552028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.551874    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"
Jul 21 12:54:55.552028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.551913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr" event={"ID":"d7bc63a5-3fa5-43c2-b059-4770e55053b5","Type":"ContainerDied","Data":"6ec9a9bb233dc71794358a8e4930493f0dff7f72605a6b23cecdac992e042cb6"}
Jul 21 12:54:55.552028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.551936    2562 scope.go:117] "RemoveContainer" containerID="8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a"
Jul 21 12:54:55.563245 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.563226    2562 scope.go:117] "RemoveContainer" containerID="8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a"
Jul 21 12:54:55.563507 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:54:55.563488    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a\": container with ID starting with 8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a not found: ID does not exist" containerID="8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a"
Jul 21 12:54:55.563555 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.563518    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a"} err="failed to get container status \"8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a\": rpc error: code = NotFound desc = could not find container \"8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a\": container with ID starting with 8d7765c9fb80e12b0ee740c54ab20489f5df876112bd8b0a9cebb01c47f8f90a not found: ID does not exist"
Jul 21 12:54:55.580440 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.580412    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"]
Jul 21 12:54:55.585269 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:55.585245    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8af997-66b9b7b755-5gdxr"]
Jul 21 12:54:56.417161 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:56.417131    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7bc63a5-3fa5-43c2-b059-4770e55053b5" path="/var/lib/kubelet/pods/d7bc63a5-3fa5-43c2-b059-4770e55053b5/volumes"
Jul 21 12:54:59.601217 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:54:59.601183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:55:06.077419 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.077381    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"]
Jul 21 12:55:06.077941 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.077652    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy" containerID="cri-o://951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab" gracePeriod=30
Jul 21 12:55:06.945683 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.945649    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"]
Jul 21 12:55:06.945995 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.945981    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy"
Jul 21 12:55:06.946066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.945998    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy"
Jul 21 12:55:06.946066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.946017    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4" containerName="maas-api"
Jul 21 12:55:06.946066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.946025    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4" containerName="maas-api"
Jul 21 12:55:06.946066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.946042    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bc63a5-3fa5-43c2-b059-4770e55053b5" containerName="maas-api"
Jul 21 12:55:06.946066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.946050    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bc63a5-3fa5-43c2-b059-4770e55053b5" containerName="maas-api"
Jul 21 12:55:06.946313 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.946101    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy"
Jul 21 12:55:06.946313 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.946110    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7bc63a5-3fa5-43c2-b059-4770e55053b5" containerName="maas-api"
Jul 21 12:55:06.946313 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.946116    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3797b4a4-ad6f-480f-ae60-5b0ae3e3a2a4" containerName="maas-api"
Jul 21 12:55:06.950390 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.950370    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:06.952716 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.952692    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-782207-openshift-default-dockercfg-zxh2f\""
Jul 21 12:55:06.964462 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:06.964435    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"]
Jul 21 12:55:07.010740 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.010704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-socket\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.010740 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.010743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlbnk\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-kube-api-access-hlbnk\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.010961 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.010761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-certs\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.010961 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.010814    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d085d6ea-147e-4295-89d8-143eec1b224f-istiod-ca-cert\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.010961 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.010839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-envoy\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.010961 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.010855    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-data\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.010961 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.010890    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d085d6ea-147e-4295-89d8-143eec1b224f-istio-podinfo\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.010961 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.010917    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-istio-token\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.010961 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.010938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-credential-socket\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.112119 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112078    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-socket\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.112597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112129    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlbnk\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-kube-api-access-hlbnk\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.112597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112160    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-certs\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.112597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112188    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d085d6ea-147e-4295-89d8-143eec1b224f-istiod-ca-cert\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.112597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-envoy\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.112597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112240    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-data\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.112597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d085d6ea-147e-4295-89d8-143eec1b224f-istio-podinfo\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.112597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-istio-token\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.112597 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112585    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-credential-socket\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.113066 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.112609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-socket\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.113125 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.113092    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d085d6ea-147e-4295-89d8-143eec1b224f-istiod-ca-cert\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.113232 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.113209    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-certs\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.113437 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.113415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-data\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.113514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.113469    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-credential-socket\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.114740 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.114720    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-envoy\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.115058 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.115038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d085d6ea-147e-4295-89d8-143eec1b224f-istio-podinfo\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.121648 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.121624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-istio-token\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.121648 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.121643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlbnk\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-kube-api-access-hlbnk\") pod \"e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") " pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.261202 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.261113    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:07.399950 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.399919    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"]
Jul 21 12:55:07.401692 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:55:07.401666    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd085d6ea_147e_4295_89d8_143eec1b224f.slice/crio-263367c351b45b37573c1e7c9ae901bc1ff2e28f7e5a6a8b31eeb9cab7a958bb WatchSource:0}: Error finding container 263367c351b45b37573c1e7c9ae901bc1ff2e28f7e5a6a8b31eeb9cab7a958bb: Status 404 returned error can't find the container with id 263367c351b45b37573c1e7c9ae901bc1ff2e28f7e5a6a8b31eeb9cab7a958bb
Jul 21 12:55:07.403731 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.403698    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:07.403857 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.403762    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:07.403857 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.403817    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:07.613481 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.613446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" event={"ID":"d085d6ea-147e-4295-89d8-143eec1b224f","Type":"ContainerStarted","Data":"c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440"}
Jul 21 12:55:07.613668 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.613506    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" event={"ID":"d085d6ea-147e-4295-89d8-143eec1b224f","Type":"ContainerStarted","Data":"263367c351b45b37573c1e7c9ae901bc1ff2e28f7e5a6a8b31eeb9cab7a958bb"}
Jul 21 12:55:07.638759 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:07.638703    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" podStartSLOduration=1.6386845380000001 podStartE2EDuration="1.638684538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:55:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:55:07.636886326 +0000 UTC m=+2631.695317330" watchObservedRunningTime="2026-07-21 12:55:07.638684538 +0000 UTC m=+2631.697115542"
Jul 21 12:55:08.261498 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:08.261460    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:08.266789 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:08.266743    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:08.618817 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:08.618759    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:08.619796 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:08.619754    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:09.658046 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.658011    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"]
Jul 21 12:55:09.658421 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.658353    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy"
Jul 21 12:55:09.658421 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.658363    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy"
Jul 21 12:55:09.658421 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.658419    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="23be5521-4120-4dba-afef-150d3ecf1b8d" containerName="istio-proxy"
Jul 21 12:55:09.661285 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.661267    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:09.664470 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.664440    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\""
Jul 21 12:55:09.664470 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.664464    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-782207-serving-cert\""
Jul 21 12:55:09.664974 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.664953    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:55:09.665065 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.664962    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:55:09.673813 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.673750    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"]
Jul 21 12:55:09.736270 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.736232    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qml7\" (UniqueName: \"kubernetes.io/projected/23fbd862-29a8-4376-9b30-9cab8122e30e-kube-api-access-6qml7\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:09.736432 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.736291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23fbd862-29a8-4376-9b30-9cab8122e30e-maas-api-tls\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:09.736432 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.736363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23fbd862-29a8-4376-9b30-9cab8122e30e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:09.837927 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.837892    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23fbd862-29a8-4376-9b30-9cab8122e30e-maas-api-tls\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:09.838100 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.837937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23fbd862-29a8-4376-9b30-9cab8122e30e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:09.838100 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.838020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qml7\" (UniqueName: \"kubernetes.io/projected/23fbd862-29a8-4376-9b30-9cab8122e30e-kube-api-access-6qml7\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:09.838100 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:55:09.838051    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-serving-cert: secret "maas-api-e2e-shared-a-782207-serving-cert" not found
Jul 21 12:55:09.838213 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:55:09.838130    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23fbd862-29a8-4376-9b30-9cab8122e30e-maas-api-tls podName:23fbd862-29a8-4376-9b30-9cab8122e30e nodeName:}" failed. No retries permitted until 2026-07-21 12:55:10.338108913 +0000 UTC m=+2634.396539919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23fbd862-29a8-4376-9b30-9cab8122e30e-maas-api-tls") pod "maas-api-e2e-shared-a-782207-76dd784c89-mr22j" (UID: "23fbd862-29a8-4376-9b30-9cab8122e30e") : secret "maas-api-e2e-shared-a-782207-serving-cert" not found
Jul 21 12:55:09.838790 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.838754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23fbd862-29a8-4376-9b30-9cab8122e30e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:09.849324 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:09.849267    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qml7\" (UniqueName: \"kubernetes.io/projected/23fbd862-29a8-4376-9b30-9cab8122e30e-kube-api-access-6qml7\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:10.342024 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:10.341982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23fbd862-29a8-4376-9b30-9cab8122e30e-maas-api-tls\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:10.344605 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:10.344586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23fbd862-29a8-4376-9b30-9cab8122e30e-maas-api-tls\") pod \"maas-api-e2e-shared-a-782207-76dd784c89-mr22j\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:10.572603 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:10.572563    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:10.710336 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:10.710296    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"]
Jul 21 12:55:10.713261 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:55:10.713235    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23fbd862_29a8_4376_9b30_9cab8122e30e.slice/crio-969efb617b03737a749bc7339a0ae5ff5870bc17c82b286ec387f99bbec9f734 WatchSource:0}: Error finding container 969efb617b03737a749bc7339a0ae5ff5870bc17c82b286ec387f99bbec9f734: Status 404 returned error can't find the container with id 969efb617b03737a749bc7339a0ae5ff5870bc17c82b286ec387f99bbec9f734
Jul 21 12:55:11.332186 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.332161    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:55:11.352004 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.351933    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-credential-socket\") pod \"f648e020-c419-4dcb-9ae2-9fda542fee02\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") "
Jul 21 12:55:11.352004 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.351964    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-podinfo\") pod \"f648e020-c419-4dcb-9ae2-9fda542fee02\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") "
Jul 21 12:55:11.352244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352011    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f648e020-c419-4dcb-9ae2-9fda542fee02-istiod-ca-cert\") pod \"f648e020-c419-4dcb-9ae2-9fda542fee02\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") "
Jul 21 12:55:11.352244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352029    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-envoy\") pod \"f648e020-c419-4dcb-9ae2-9fda542fee02\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") "
Jul 21 12:55:11.352244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352063    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-data\") pod \"f648e020-c419-4dcb-9ae2-9fda542fee02\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") "
Jul 21 12:55:11.352244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352088    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-socket\") pod \"f648e020-c419-4dcb-9ae2-9fda542fee02\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") "
Jul 21 12:55:11.352244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352121    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6l77v\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-kube-api-access-6l77v\") pod \"f648e020-c419-4dcb-9ae2-9fda542fee02\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") "
Jul 21 12:55:11.352244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352150    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-certs\") pod \"f648e020-c419-4dcb-9ae2-9fda542fee02\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") "
Jul 21 12:55:11.352244 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352174    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-token\") pod \"f648e020-c419-4dcb-9ae2-9fda542fee02\" (UID: \"f648e020-c419-4dcb-9ae2-9fda542fee02\") "
Jul 21 12:55:11.352688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352237    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f648e020-c419-4dcb-9ae2-9fda542fee02" (UID: "f648e020-c419-4dcb-9ae2-9fda542fee02"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:11.352688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352471    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:55:11.352688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352524    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f648e020-c419-4dcb-9ae2-9fda542fee02" (UID: "f648e020-c419-4dcb-9ae2-9fda542fee02"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:11.352688 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.352584    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f648e020-c419-4dcb-9ae2-9fda542fee02-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f648e020-c419-4dcb-9ae2-9fda542fee02" (UID: "f648e020-c419-4dcb-9ae2-9fda542fee02"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:55:11.353602 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.353290    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-data" (OuterVolumeSpecName: "istio-data") pod "f648e020-c419-4dcb-9ae2-9fda542fee02" (UID: "f648e020-c419-4dcb-9ae2-9fda542fee02"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:11.353602 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.353413    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f648e020-c419-4dcb-9ae2-9fda542fee02" (UID: "f648e020-c419-4dcb-9ae2-9fda542fee02"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:11.354643 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.354621    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-token" (OuterVolumeSpecName: "istio-token") pod "f648e020-c419-4dcb-9ae2-9fda542fee02" (UID: "f648e020-c419-4dcb-9ae2-9fda542fee02"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:55:11.354891 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.354869    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-kube-api-access-6l77v" (OuterVolumeSpecName: "kube-api-access-6l77v") pod "f648e020-c419-4dcb-9ae2-9fda542fee02" (UID: "f648e020-c419-4dcb-9ae2-9fda542fee02"). InnerVolumeSpecName "kube-api-access-6l77v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:55:11.355135 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.355119    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f648e020-c419-4dcb-9ae2-9fda542fee02" (UID: "f648e020-c419-4dcb-9ae2-9fda542fee02"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:55:11.355430 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.355414    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f648e020-c419-4dcb-9ae2-9fda542fee02" (UID: "f648e020-c419-4dcb-9ae2-9fda542fee02"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:11.452914 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.452883    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:55:11.452914 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.452911    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:55:11.452914 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.452922    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:55:11.453198 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.452935    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l77v\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-kube-api-access-6l77v\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:55:11.453198 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.452947    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f648e020-c419-4dcb-9ae2-9fda542fee02-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:55:11.453198 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.452958    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:55:11.453198 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.452969    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f648e020-c419-4dcb-9ae2-9fda542fee02-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:55:11.453198 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.452979    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f648e020-c419-4dcb-9ae2-9fda542fee02-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 12:55:11.637655 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.637549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j" event={"ID":"23fbd862-29a8-4376-9b30-9cab8122e30e","Type":"ContainerStarted","Data":"5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599"}
Jul 21 12:55:11.637655 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.637619    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:11.637655 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.637647    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j" event={"ID":"23fbd862-29a8-4376-9b30-9cab8122e30e","Type":"ContainerStarted","Data":"969efb617b03737a749bc7339a0ae5ff5870bc17c82b286ec387f99bbec9f734"}
Jul 21 12:55:11.639409 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.639379    2562 generic.go:356] "Generic (PLEG): container finished" podID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerID="951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab" exitCode=0
Jul 21 12:55:11.639528 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.639429    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"
Jul 21 12:55:11.639528 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.639432    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" event={"ID":"f648e020-c419-4dcb-9ae2-9fda542fee02","Type":"ContainerDied","Data":"951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab"}
Jul 21 12:55:11.639601 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.639541    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq" event={"ID":"f648e020-c419-4dcb-9ae2-9fda542fee02","Type":"ContainerDied","Data":"f1f25f9150acffbe859a6d2c27cfebbd321639715def79022cf99c28a45c4e2f"}
Jul 21 12:55:11.639601 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.639568    2562 scope.go:117] "RemoveContainer" containerID="951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab"
Jul 21 12:55:11.652878 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.652858    2562 scope.go:117] "RemoveContainer" containerID="0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17"
Jul 21 12:55:11.657713 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.657664    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j" podStartSLOduration=2.1641352879999998 podStartE2EDuration="2.657652214s" totalImagesPullingTime="493.516926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:55:11.655552074 +0000 UTC m=+2635.713983113" watchObservedRunningTime="2026-07-21 12:55:11.657652214 +0000 UTC m=+2635.716083218"
Jul 21 12:55:11.661564 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.661547    2562 scope.go:117] "RemoveContainer" containerID="951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab"
Jul 21 12:55:11.661808 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:55:11.661769    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab\": container with ID starting with 951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab not found: ID does not exist" containerID="951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab"
Jul 21 12:55:11.661905 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.661812    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab"} err="failed to get container status \"951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab\": rpc error: code = NotFound desc = could not find container \"951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab\": container with ID starting with 951865a15ba4459e73cc1fcdb21ebbaf3ac461f966c013460c123069e2bf9bab not found: ID does not exist"
Jul 21 12:55:11.661905 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.661829    2562 scope.go:117] "RemoveContainer" containerID="0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17"
Jul 21 12:55:11.662067 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:55:11.662046    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17\": container with ID starting with 0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17 not found: ID does not exist" containerID="0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17"
Jul 21 12:55:11.662104 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.662074    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17"} err="failed to get container status \"0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17\": rpc error: code = NotFound desc = could not find container \"0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17\": container with ID starting with 0897ce152cc2906ea01947ee8d4ca01dadd032649e41003e281d8cafdd3e8c17 not found: ID does not exist"
Jul 21 12:55:11.673503 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.673479    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"]
Jul 21 12:55:11.680157 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:11.680136    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8af997-openshift-default-769bdc484f-r7knq"]
Jul 21 12:55:12.417358 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:12.417314    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" path="/var/lib/kubelet/pods/f648e020-c419-4dcb-9ae2-9fda542fee02/volumes"
Jul 21 12:55:14.095343 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.095300    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:55:14.639789 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.639738    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"]
Jul 21 12:55:14.640200 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.640181    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy"
Jul 21 12:55:14.640200 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.640200    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy"
Jul 21 12:55:14.640380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.640284    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy"
Jul 21 12:55:14.640380 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.640297    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy"
Jul 21 12:55:14.644748 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.644727    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.647470 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.647449    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-48842b-openshift-default-dockercfg-zdgh8\""
Jul 21 12:55:14.655563 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.655536    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"]
Jul 21 12:55:14.680757 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.680721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hvnd\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-kube-api-access-8hvnd\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.680912 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.680801    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-socket\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.680912 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.680844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8352ba85-d596-4127-8ee5-c7922f885924-istio-podinfo\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.681085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.680913    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-credential-socket\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.681085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.680942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-istio-token\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.681085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.680970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8352ba85-d596-4127-8ee5-c7922f885924-istiod-ca-cert\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.681085 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.680998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-envoy\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.681282 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.681146    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-certs\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.681282 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.681183    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-data\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782231 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-envoy\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782410 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-certs\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782410 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-data\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782542 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hvnd\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-kube-api-access-8hvnd\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782542 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-socket\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782542 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8352ba85-d596-4127-8ee5-c7922f885924-istio-podinfo\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782695 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-certs\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782695 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-credential-socket\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782695 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-istio-token\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782695 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8352ba85-d596-4127-8ee5-c7922f885924-istiod-ca-cert\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782944 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782699    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-data\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.782944 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.782821    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-socket\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.783161 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.783133    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-credential-socket\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.783407 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.783384    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8352ba85-d596-4127-8ee5-c7922f885924-istiod-ca-cert\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.784689 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.784670    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-envoy\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.785082 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.785060    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8352ba85-d596-4127-8ee5-c7922f885924-istio-podinfo\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.790954 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.790931    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-istio-token\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.791123 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.791104    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hvnd\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-kube-api-access-8hvnd\") pod \"e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") " pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:14.957175 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:14.957083    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:15.103157 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:15.103130    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"]
Jul 21 12:55:15.104732 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:55:15.104708    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8352ba85_d596_4127_8ee5_c7922f885924.slice/crio-3b9fe3a4f33c73d83be1a2b8ba0108b85119d84788d8d0a6b6d1a893b4df8822 WatchSource:0}: Error finding container 3b9fe3a4f33c73d83be1a2b8ba0108b85119d84788d8d0a6b6d1a893b4df8822: Status 404 returned error can't find the container with id 3b9fe3a4f33c73d83be1a2b8ba0108b85119d84788d8d0a6b6d1a893b4df8822
Jul 21 12:55:15.106866 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:15.106831    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:15.106948 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:15.106917    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:15.106988 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:15.106950    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:15.671596 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:15.671556    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" event={"ID":"8352ba85-d596-4127-8ee5-c7922f885924","Type":"ContainerStarted","Data":"2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8"}
Jul 21 12:55:15.671759 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:15.671608    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" event={"ID":"8352ba85-d596-4127-8ee5-c7922f885924","Type":"ContainerStarted","Data":"3b9fe3a4f33c73d83be1a2b8ba0108b85119d84788d8d0a6b6d1a893b4df8822"}
Jul 21 12:55:15.698386 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:15.698326    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" podStartSLOduration=1.698306223 podStartE2EDuration="1.698306223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:55:15.698297111 +0000 UTC m=+2639.756728116" watchObservedRunningTime="2026-07-21 12:55:15.698306223 +0000 UTC m=+2639.756737227"
Jul 21 12:55:15.958250 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:15.958138    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:16.965666 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:16.965629    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:17.664746 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.664066    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 12:55:17.681656 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.681616    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:17.682689 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.682656    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:17.761555 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.761504    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"]
Jul 21 12:55:17.762232 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.762214    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy"
Jul 21 12:55:17.762346 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.762336    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f648e020-c419-4dcb-9ae2-9fda542fee02" containerName="istio-proxy"
Jul 21 12:55:17.777695 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.777668    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:17.781341 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.781270    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"]
Jul 21 12:55:17.781568 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.781543    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-48842b-serving-cert\""
Jul 21 12:55:17.810044 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.810010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zmhj\" (UniqueName: \"kubernetes.io/projected/d91abf12-e61a-4c2a-803e-e1b0d304a469-kube-api-access-8zmhj\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:17.810195 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.810049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:17.810195 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.810067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91abf12-e61a-4c2a-803e-e1b0d304a469-openshift-service-ca\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:17.910827 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.910784    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zmhj\" (UniqueName: \"kubernetes.io/projected/d91abf12-e61a-4c2a-803e-e1b0d304a469-kube-api-access-8zmhj\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:17.911024 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.910842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:17.911024 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.910864    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91abf12-e61a-4c2a-803e-e1b0d304a469-openshift-service-ca\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:17.911024 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:55:17.910998    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-serving-cert: secret "maas-api-e2e-shared-b-48842b-serving-cert" not found
Jul 21 12:55:17.911153 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:55:17.911071    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls podName:d91abf12-e61a-4c2a-803e-e1b0d304a469 nodeName:}" failed. No retries permitted until 2026-07-21 12:55:18.411049806 +0000 UTC m=+2642.469480808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls") pod "maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22" (UID: "d91abf12-e61a-4c2a-803e-e1b0d304a469") : secret "maas-api-e2e-shared-b-48842b-serving-cert" not found
Jul 21 12:55:17.911624 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.911606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91abf12-e61a-4c2a-803e-e1b0d304a469-openshift-service-ca\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:17.949460 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:17.949389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zmhj\" (UniqueName: \"kubernetes.io/projected/d91abf12-e61a-4c2a-803e-e1b0d304a469-kube-api-access-8zmhj\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:18.416022 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:18.415986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:18.416508 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:55:18.416187    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-serving-cert: secret "maas-api-e2e-shared-b-48842b-serving-cert" not found
Jul 21 12:55:18.416508 ip-10-0-138-237 kubenswrapper[2562]: E0721 12:55:18.416255    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls podName:d91abf12-e61a-4c2a-803e-e1b0d304a469 nodeName:}" failed. No retries permitted until 2026-07-21 12:55:19.416235045 +0000 UTC m=+2643.474666032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls") pod "maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22" (UID: "d91abf12-e61a-4c2a-803e-e1b0d304a469") : secret "maas-api-e2e-shared-b-48842b-serving-cert" not found
Jul 21 12:55:19.427111 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:19.427071    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:19.429816 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:19.429769    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls\") pod \"maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:19.598968 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:19.598929    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:19.775252 ip-10-0-138-237 kubenswrapper[2562]: W0721 12:55:19.775225    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91abf12_e61a_4c2a_803e_e1b0d304a469.slice/crio-7c02a0801fc5d1b7ce87426bd32ca9adb800d9343f4acbc09f91f6662b9bf981 WatchSource:0}: Error finding container 7c02a0801fc5d1b7ce87426bd32ca9adb800d9343f4acbc09f91f6662b9bf981: Status 404 returned error can't find the container with id 7c02a0801fc5d1b7ce87426bd32ca9adb800d9343f4acbc09f91f6662b9bf981
Jul 21 12:55:19.776697 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:19.776667    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"]
Jul 21 12:55:20.703328 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:20.703287    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22" event={"ID":"d91abf12-e61a-4c2a-803e-e1b0d304a469","Type":"ContainerStarted","Data":"8dc6a3010b8e981bdd9eca038910fdcdb03b81d31f380d5e64a2fcc8164ff59a"}
Jul 21 12:55:20.703851 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:20.703342    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22" event={"ID":"d91abf12-e61a-4c2a-803e-e1b0d304a469","Type":"ContainerStarted","Data":"7c02a0801fc5d1b7ce87426bd32ca9adb800d9343f4acbc09f91f6662b9bf981"}
Jul 21 12:55:20.703851 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:20.703390    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:20.722358 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:20.722313    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22" podStartSLOduration=3.426276471 podStartE2EDuration="3.722298117s" totalImagesPullingTime="296.021646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:55:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:55:20.721559559 +0000 UTC m=+2644.779990565" watchObservedRunningTime="2026-07-21 12:55:20.722298117 +0000 UTC m=+2644.780729121"
Jul 21 12:55:22.695684 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:22.695646    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:55:22.714514 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:22.714479    2562 generic.go:356] "Generic (PLEG): container finished" podID="d085d6ea-147e-4295-89d8-143eec1b224f" containerID="c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440" exitCode=0
Jul 21 12:55:22.714671 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:22.714556    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" event={"ID":"d085d6ea-147e-4295-89d8-143eec1b224f","Type":"ContainerDied","Data":"c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440"}
Jul 21 12:55:22.714967 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:22.714953    2562 scope.go:117] "RemoveContainer" containerID="c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440"
Jul 21 12:55:22.715924 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:22.715897    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:22.715989 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:22.715973    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:22.716028 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:22.716019    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:23.618960 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:23.618922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:23.722570 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:23.722531    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" event={"ID":"d085d6ea-147e-4295-89d8-143eec1b224f","Type":"ContainerStarted","Data":"a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943"}
Jul 21 12:55:24.263276 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:24.263244    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:24.263654 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:24.263634    2562 patch_prober.go:28] interesting pod/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 21 12:55:24.263835 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:24.263812    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 21 12:55:25.261657 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:25.261619    2562 patch_prober.go:28] interesting pod/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 21 12:55:25.262662 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:25.262257    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 21 12:55:26.261590 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:26.261540    2562 patch_prober.go:28] interesting pod/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 21 12:55:26.261858 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:26.261614    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 21 12:55:26.714229 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:26.714197    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 12:55:27.261727 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:27.261687    2562 patch_prober.go:28] interesting pod/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 21 12:55:27.262044 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:27.262011    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 21 12:55:28.265515 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:28.265428    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:28.265978 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:28.265750    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:28.266386 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:28.266368    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 12:55:30.761163 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:30.761122    2562 generic.go:356] "Generic (PLEG): container finished" podID="8352ba85-d596-4127-8ee5-c7922f885924" containerID="2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8" exitCode=0
Jul 21 12:55:30.761572 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:30.761189    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" event={"ID":"8352ba85-d596-4127-8ee5-c7922f885924","Type":"ContainerDied","Data":"2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8"}
Jul 21 12:55:30.761616 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:30.761577    2562 scope.go:117] "RemoveContainer" containerID="2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8"
Jul 21 12:55:30.762625 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:30.762599    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:30.762696 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:30.762686    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:30.762739 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:30.762723    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:55:31.768888 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:31.768852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" event={"ID":"8352ba85-d596-4127-8ee5-c7922f885924","Type":"ContainerStarted","Data":"11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49"}
Jul 21 12:55:31.959171 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:31.959137    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:31.960199 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:31.960166    2562 patch_prober.go:28] interesting pod/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 21 12:55:31.960439 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:31.960413    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 21 12:55:32.681616 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:32.681583    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:32.957682 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:32.957576    2562 patch_prober.go:28] interesting pod/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 21 12:55:32.957682 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:32.957648    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 21 12:55:33.958416 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:33.958366    2562 patch_prober.go:28] interesting pod/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 21 12:55:33.958949 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:33.958442    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 21 12:55:34.958206 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:34.958166    2562 patch_prober.go:28] interesting pod/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 21 12:55:34.958518 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:34.958486    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 21 12:55:35.094535 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:35.094497    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:55:35.961599 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:35.961571    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:55:35.962496 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:55:35.962477    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 12:56:16.814130 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:56:16.814044    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:56:30.007821 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:56:30.007768    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:56:44.608874 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:56:44.608839    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:57:04.000413 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:57:04.000369    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:57:25.200535 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:57:25.200498    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:57:39.728415 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:57:39.728322    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:57:55.894443 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:57:55.894403    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:58:14.613129 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:58:14.613091    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:58:32.302205 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:58:32.302168    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:58:50.315714 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:58:50.315672    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:59:18.102663 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:59:18.102563    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:59:37.904292 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:59:37.904252    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 12:59:51.292251 ip-10-0-138-237 kubenswrapper[2562]: I0721 12:59:51.292217    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 13:00:14.596198 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:00:14.596164    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 13:00:40.602394 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:00:40.602315    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 13:00:56.109806 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:00:56.109738    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 13:01:10.591720 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:10.591686    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 13:01:33.571203 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:33.571167    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"]
Jul 21 13:01:33.571803 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:33.571449    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j" podUID="23fbd862-29a8-4376-9b30-9cab8122e30e" containerName="maas-api" containerID="cri-o://5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599" gracePeriod=30
Jul 21 13:01:34.015546 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.015516    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 13:01:34.071895 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.071863    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23fbd862-29a8-4376-9b30-9cab8122e30e-openshift-service-ca\") pod \"23fbd862-29a8-4376-9b30-9cab8122e30e\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") "
Jul 21 13:01:34.072058 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.071907    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23fbd862-29a8-4376-9b30-9cab8122e30e-maas-api-tls\") pod \"23fbd862-29a8-4376-9b30-9cab8122e30e\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") "
Jul 21 13:01:34.072058 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.071945    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qml7\" (UniqueName: \"kubernetes.io/projected/23fbd862-29a8-4376-9b30-9cab8122e30e-kube-api-access-6qml7\") pod \"23fbd862-29a8-4376-9b30-9cab8122e30e\" (UID: \"23fbd862-29a8-4376-9b30-9cab8122e30e\") "
Jul 21 13:01:34.072257 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.072229    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23fbd862-29a8-4376-9b30-9cab8122e30e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "23fbd862-29a8-4376-9b30-9cab8122e30e" (UID: "23fbd862-29a8-4376-9b30-9cab8122e30e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:01:34.074200 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.074172    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23fbd862-29a8-4376-9b30-9cab8122e30e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "23fbd862-29a8-4376-9b30-9cab8122e30e" (UID: "23fbd862-29a8-4376-9b30-9cab8122e30e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:01:34.074302 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.074209    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23fbd862-29a8-4376-9b30-9cab8122e30e-kube-api-access-6qml7" (OuterVolumeSpecName: "kube-api-access-6qml7") pod "23fbd862-29a8-4376-9b30-9cab8122e30e" (UID: "23fbd862-29a8-4376-9b30-9cab8122e30e"). InnerVolumeSpecName "kube-api-access-6qml7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:01:34.172829 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.172709    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23fbd862-29a8-4376-9b30-9cab8122e30e-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:34.172829 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.172740    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23fbd862-29a8-4376-9b30-9cab8122e30e-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:34.172829 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.172749    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qml7\" (UniqueName: \"kubernetes.io/projected/23fbd862-29a8-4376-9b30-9cab8122e30e-kube-api-access-6qml7\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:34.595344 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.595310    2562 generic.go:356] "Generic (PLEG): container finished" podID="23fbd862-29a8-4376-9b30-9cab8122e30e" containerID="5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599" exitCode=0
Jul 21 13:01:34.595867 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.595352    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"
Jul 21 13:01:34.595867 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.595386    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j" event={"ID":"23fbd862-29a8-4376-9b30-9cab8122e30e","Type":"ContainerDied","Data":"5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599"}
Jul 21 13:01:34.595867 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.595444    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j" event={"ID":"23fbd862-29a8-4376-9b30-9cab8122e30e","Type":"ContainerDied","Data":"969efb617b03737a749bc7339a0ae5ff5870bc17c82b286ec387f99bbec9f734"}
Jul 21 13:01:34.595867 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.595470    2562 scope.go:117] "RemoveContainer" containerID="5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599"
Jul 21 13:01:34.604091 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.604071    2562 scope.go:117] "RemoveContainer" containerID="5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599"
Jul 21 13:01:34.604333 ip-10-0-138-237 kubenswrapper[2562]: E0721 13:01:34.604314    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599\": container with ID starting with 5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599 not found: ID does not exist" containerID="5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599"
Jul 21 13:01:34.604386 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.604344    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599"} err="failed to get container status \"5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599\": rpc error: code = NotFound desc = could not find container \"5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599\": container with ID starting with 5c51daa5eb41310459fb595cd89e018eef2630c32da974349250fd4c63c77599 not found: ID does not exist"
Jul 21 13:01:34.612450 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.612422    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"]
Jul 21 13:01:34.616419 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:34.616398    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-782207-76dd784c89-mr22j"]
Jul 21 13:01:36.417821 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:36.417764    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23fbd862-29a8-4376-9b30-9cab8122e30e" path="/var/lib/kubelet/pods/23fbd862-29a8-4376-9b30-9cab8122e30e/volumes"
Jul 21 13:01:36.795977 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:36.795892    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 13:01:44.568979 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:44.568942    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"]
Jul 21 13:01:44.646822 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:44.646722    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy" containerID="cri-o://a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943" gracePeriod=30
Jul 21 13:01:49.900495 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:49.900470    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 13:01:50.015623 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015517    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-credential-socket\") pod \"d085d6ea-147e-4295-89d8-143eec1b224f\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") "
Jul 21 13:01:50.015623 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015567    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlbnk\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-kube-api-access-hlbnk\") pod \"d085d6ea-147e-4295-89d8-143eec1b224f\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") "
Jul 21 13:01:50.015623 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015585    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-envoy\") pod \"d085d6ea-147e-4295-89d8-143eec1b224f\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") "
Jul 21 13:01:50.015623 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015605    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d085d6ea-147e-4295-89d8-143eec1b224f-istio-podinfo\") pod \"d085d6ea-147e-4295-89d8-143eec1b224f\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") "
Jul 21 13:01:50.015623 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015624    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-socket\") pod \"d085d6ea-147e-4295-89d8-143eec1b224f\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") "
Jul 21 13:01:50.016065 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015639    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-certs\") pod \"d085d6ea-147e-4295-89d8-143eec1b224f\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") "
Jul 21 13:01:50.016065 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d085d6ea-147e-4295-89d8-143eec1b224f-istiod-ca-cert\") pod \"d085d6ea-147e-4295-89d8-143eec1b224f\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") "
Jul 21 13:01:50.016065 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015723    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-data\") pod \"d085d6ea-147e-4295-89d8-143eec1b224f\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") "
Jul 21 13:01:50.016065 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015949    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d085d6ea-147e-4295-89d8-143eec1b224f" (UID: "d085d6ea-147e-4295-89d8-143eec1b224f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:01:50.016065 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015965    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d085d6ea-147e-4295-89d8-143eec1b224f" (UID: "d085d6ea-147e-4295-89d8-143eec1b224f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:01:50.016065 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.016044    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d085d6ea-147e-4295-89d8-143eec1b224f" (UID: "d085d6ea-147e-4295-89d8-143eec1b224f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:01:50.016530 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.015749    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-istio-token\") pod \"d085d6ea-147e-4295-89d8-143eec1b224f\" (UID: \"d085d6ea-147e-4295-89d8-143eec1b224f\") "
Jul 21 13:01:50.016530 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.016150    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d085d6ea-147e-4295-89d8-143eec1b224f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d085d6ea-147e-4295-89d8-143eec1b224f" (UID: "d085d6ea-147e-4295-89d8-143eec1b224f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:01:50.016530 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.016382    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:50.016530 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.016397    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:50.016530 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.016408    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:50.016530 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.016418    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d085d6ea-147e-4295-89d8-143eec1b224f-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:50.016966 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.016938    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d085d6ea-147e-4295-89d8-143eec1b224f" (UID: "d085d6ea-147e-4295-89d8-143eec1b224f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:01:50.018091 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.018058    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d085d6ea-147e-4295-89d8-143eec1b224f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d085d6ea-147e-4295-89d8-143eec1b224f" (UID: "d085d6ea-147e-4295-89d8-143eec1b224f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:01:50.018206 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.018110    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d085d6ea-147e-4295-89d8-143eec1b224f" (UID: "d085d6ea-147e-4295-89d8-143eec1b224f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:01:50.018206 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.018133    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-kube-api-access-hlbnk" (OuterVolumeSpecName: "kube-api-access-hlbnk") pod "d085d6ea-147e-4295-89d8-143eec1b224f" (UID: "d085d6ea-147e-4295-89d8-143eec1b224f"). InnerVolumeSpecName "kube-api-access-hlbnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:01:50.018331 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.018303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d085d6ea-147e-4295-89d8-143eec1b224f" (UID: "d085d6ea-147e-4295-89d8-143eec1b224f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:01:50.117815 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.117753    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:50.117815 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.117807    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:50.117815 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.117817    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlbnk\" (UniqueName: \"kubernetes.io/projected/d085d6ea-147e-4295-89d8-143eec1b224f-kube-api-access-hlbnk\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:50.117815 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.117827    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d085d6ea-147e-4295-89d8-143eec1b224f-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:50.118086 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.117835    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d085d6ea-147e-4295-89d8-143eec1b224f-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:50.683923 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.683829    2562 generic.go:356] "Generic (PLEG): container finished" podID="d085d6ea-147e-4295-89d8-143eec1b224f" containerID="a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943" exitCode=0
Jul 21 13:01:50.683923 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.683881    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"
Jul 21 13:01:50.683923 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.683897    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" event={"ID":"d085d6ea-147e-4295-89d8-143eec1b224f","Type":"ContainerDied","Data":"a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943"}
Jul 21 13:01:50.684193 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.683952    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq" event={"ID":"d085d6ea-147e-4295-89d8-143eec1b224f","Type":"ContainerDied","Data":"263367c351b45b37573c1e7c9ae901bc1ff2e28f7e5a6a8b31eeb9cab7a958bb"}
Jul 21 13:01:50.684193 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.683979    2562 scope.go:117] "RemoveContainer" containerID="a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943"
Jul 21 13:01:50.692945 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.692922    2562 scope.go:117] "RemoveContainer" containerID="c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440"
Jul 21 13:01:50.702354 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.702329    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"]
Jul 21 13:01:50.702653 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.702620    2562 scope.go:117] "RemoveContainer" containerID="a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943"
Jul 21 13:01:50.707376 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.707348    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-782207-openshift-default-cfc65b4ff-cc8xq"]
Jul 21 13:01:50.707510 ip-10-0-138-237 kubenswrapper[2562]: E0721 13:01:50.707459    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943\": container with ID starting with a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943 not found: ID does not exist" containerID="a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943"
Jul 21 13:01:50.707510 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.707486    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943"} err="failed to get container status \"a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943\": rpc error: code = NotFound desc = could not find container \"a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943\": container with ID starting with a7bcdf8cf859dde2846a28ed99468b01876dcdee188ada748b1831aa0ff29943 not found: ID does not exist"
Jul 21 13:01:50.707510 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.707507    2562 scope.go:117] "RemoveContainer" containerID="c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440"
Jul 21 13:01:50.707856 ip-10-0-138-237 kubenswrapper[2562]: E0721 13:01:50.707827    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440\": container with ID starting with c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440 not found: ID does not exist" containerID="c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440"
Jul 21 13:01:50.707965 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:50.707864    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440"} err="failed to get container status \"c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440\": rpc error: code = NotFound desc = could not find container \"c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440\": container with ID starting with c729271235fd4bbcc41e1f8c18a5b2000cbfc1a1ba13d902df6ffc7f754be440 not found: ID does not exist"
Jul 21 13:01:52.417674 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:52.417634    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" path="/var/lib/kubelet/pods/d085d6ea-147e-4295-89d8-143eec1b224f/volumes"
Jul 21 13:01:55.539378 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.539342    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"]
Jul 21 13:01:55.539809 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.539530    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22" podUID="d91abf12-e61a-4c2a-803e-e1b0d304a469" containerName="maas-api" containerID="cri-o://8dc6a3010b8e981bdd9eca038910fdcdb03b81d31f380d5e64a2fcc8164ff59a" gracePeriod=30
Jul 21 13:01:55.712084 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.712052    2562 generic.go:356] "Generic (PLEG): container finished" podID="d91abf12-e61a-4c2a-803e-e1b0d304a469" containerID="8dc6a3010b8e981bdd9eca038910fdcdb03b81d31f380d5e64a2fcc8164ff59a" exitCode=0
Jul 21 13:01:55.712250 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.712128    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22" event={"ID":"d91abf12-e61a-4c2a-803e-e1b0d304a469","Type":"ContainerDied","Data":"8dc6a3010b8e981bdd9eca038910fdcdb03b81d31f380d5e64a2fcc8164ff59a"}
Jul 21 13:01:55.789145 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.789120    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 13:01:55.870897 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.870849    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls\") pod \"d91abf12-e61a-4c2a-803e-e1b0d304a469\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") "
Jul 21 13:01:55.871087 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.870957    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zmhj\" (UniqueName: \"kubernetes.io/projected/d91abf12-e61a-4c2a-803e-e1b0d304a469-kube-api-access-8zmhj\") pod \"d91abf12-e61a-4c2a-803e-e1b0d304a469\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") "
Jul 21 13:01:55.871087 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.870986    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91abf12-e61a-4c2a-803e-e1b0d304a469-openshift-service-ca\") pod \"d91abf12-e61a-4c2a-803e-e1b0d304a469\" (UID: \"d91abf12-e61a-4c2a-803e-e1b0d304a469\") "
Jul 21 13:01:55.871397 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.871369    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d91abf12-e61a-4c2a-803e-e1b0d304a469-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d91abf12-e61a-4c2a-803e-e1b0d304a469" (UID: "d91abf12-e61a-4c2a-803e-e1b0d304a469"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:01:55.873084 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.873058    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d91abf12-e61a-4c2a-803e-e1b0d304a469" (UID: "d91abf12-e61a-4c2a-803e-e1b0d304a469"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:01:55.873255 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.873230    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d91abf12-e61a-4c2a-803e-e1b0d304a469-kube-api-access-8zmhj" (OuterVolumeSpecName: "kube-api-access-8zmhj") pod "d91abf12-e61a-4c2a-803e-e1b0d304a469" (UID: "d91abf12-e61a-4c2a-803e-e1b0d304a469"). InnerVolumeSpecName "kube-api-access-8zmhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:01:55.971748 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.971700    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91abf12-e61a-4c2a-803e-e1b0d304a469-openshift-service-ca\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:55.971748 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.971743    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d91abf12-e61a-4c2a-803e-e1b0d304a469-maas-api-tls\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:55.971748 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:55.971752    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zmhj\" (UniqueName: \"kubernetes.io/projected/d91abf12-e61a-4c2a-803e-e1b0d304a469-kube-api-access-8zmhj\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:01:56.718452 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:56.718361    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"
Jul 21 13:01:56.718452 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:56.718390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22" event={"ID":"d91abf12-e61a-4c2a-803e-e1b0d304a469","Type":"ContainerDied","Data":"7c02a0801fc5d1b7ce87426bd32ca9adb800d9343f4acbc09f91f6662b9bf981"}
Jul 21 13:01:56.718950 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:56.718455    2562 scope.go:117] "RemoveContainer" containerID="8dc6a3010b8e981bdd9eca038910fdcdb03b81d31f380d5e64a2fcc8164ff59a"
Jul 21 13:01:56.739951 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:56.739918    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"]
Jul 21 13:01:56.743520 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:56.743494    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-48842b-58fc98dc5b-spr22"]
Jul 21 13:01:58.397232 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:58.397196    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wjrqj"]
Jul 21 13:01:58.417126 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:01:58.417096    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d91abf12-e61a-4c2a-803e-e1b0d304a469" path="/var/lib/kubelet/pods/d91abf12-e61a-4c2a-803e-e1b0d304a469/volumes"
Jul 21 13:02:06.510482 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:06.510447    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"]
Jul 21 13:02:06.510992 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:06.510755    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy" containerID="cri-o://11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49" gracePeriod=30
Jul 21 13:02:11.760129 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.760103    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 13:02:11.795133 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.795054    2562 generic.go:356] "Generic (PLEG): container finished" podID="8352ba85-d596-4127-8ee5-c7922f885924" containerID="11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49" exitCode=0
Jul 21 13:02:11.795133 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.795116    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"
Jul 21 13:02:11.795322 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.795121    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" event={"ID":"8352ba85-d596-4127-8ee5-c7922f885924","Type":"ContainerDied","Data":"11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49"}
Jul 21 13:02:11.795322 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.795239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv" event={"ID":"8352ba85-d596-4127-8ee5-c7922f885924","Type":"ContainerDied","Data":"3b9fe3a4f33c73d83be1a2b8ba0108b85119d84788d8d0a6b6d1a893b4df8822"}
Jul 21 13:02:11.795322 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.795261    2562 scope.go:117] "RemoveContainer" containerID="11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49"
Jul 21 13:02:11.803971 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.803952    2562 scope.go:117] "RemoveContainer" containerID="2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8"
Jul 21 13:02:11.811915 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.811899    2562 scope.go:117] "RemoveContainer" containerID="11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49"
Jul 21 13:02:11.812165 ip-10-0-138-237 kubenswrapper[2562]: E0721 13:02:11.812145    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49\": container with ID starting with 11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49 not found: ID does not exist" containerID="11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49"
Jul 21 13:02:11.812233 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.812179    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49"} err="failed to get container status \"11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49\": rpc error: code = NotFound desc = could not find container \"11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49\": container with ID starting with 11bfa1a125881b3bab36d2f7b0005329228647fcbffe2fa446342889dcbd2b49 not found: ID does not exist"
Jul 21 13:02:11.812233 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.812204    2562 scope.go:117] "RemoveContainer" containerID="2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8"
Jul 21 13:02:11.812457 ip-10-0-138-237 kubenswrapper[2562]: E0721 13:02:11.812440    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8\": container with ID starting with 2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8 not found: ID does not exist" containerID="2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8"
Jul 21 13:02:11.812503 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.812462    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8"} err="failed to get container status \"2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8\": rpc error: code = NotFound desc = could not find container \"2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8\": container with ID starting with 2709ab78b911a58b1bbe2a56359217884504b009a623c3176b2bc9cdcf9dd4c8 not found: ID does not exist"
Jul 21 13:02:11.915043 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915009    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-istio-token\") pod \"8352ba85-d596-4127-8ee5-c7922f885924\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") "
Jul 21 13:02:11.915043 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915056    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-data\") pod \"8352ba85-d596-4127-8ee5-c7922f885924\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") "
Jul 21 13:02:11.915309 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915103    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-envoy\") pod \"8352ba85-d596-4127-8ee5-c7922f885924\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") "
Jul 21 13:02:11.915309 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915138    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-credential-socket\") pod \"8352ba85-d596-4127-8ee5-c7922f885924\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") "
Jul 21 13:02:11.915309 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915182    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hvnd\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-kube-api-access-8hvnd\") pod \"8352ba85-d596-4127-8ee5-c7922f885924\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") "
Jul 21 13:02:11.915309 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915208    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8352ba85-d596-4127-8ee5-c7922f885924-istiod-ca-cert\") pod \"8352ba85-d596-4127-8ee5-c7922f885924\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") "
Jul 21 13:02:11.915309 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915246    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8352ba85-d596-4127-8ee5-c7922f885924-istio-podinfo\") pod \"8352ba85-d596-4127-8ee5-c7922f885924\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") "
Jul 21 13:02:11.915309 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915273    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-certs\") pod \"8352ba85-d596-4127-8ee5-c7922f885924\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") "
Jul 21 13:02:11.915309 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915306    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-socket\") pod \"8352ba85-d596-4127-8ee5-c7922f885924\" (UID: \"8352ba85-d596-4127-8ee5-c7922f885924\") "
Jul 21 13:02:11.915656 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915448    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8352ba85-d596-4127-8ee5-c7922f885924" (UID: "8352ba85-d596-4127-8ee5-c7922f885924"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:11.915656 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915623    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-credential-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:02:11.915757 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915693    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8352ba85-d596-4127-8ee5-c7922f885924" (UID: "8352ba85-d596-4127-8ee5-c7922f885924"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:11.915859 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915822    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8352ba85-d596-4127-8ee5-c7922f885924" (UID: "8352ba85-d596-4127-8ee5-c7922f885924"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:11.915921 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.915867    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8352ba85-d596-4127-8ee5-c7922f885924-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8352ba85-d596-4127-8ee5-c7922f885924" (UID: "8352ba85-d596-4127-8ee5-c7922f885924"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:02:11.916153 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.916130    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-data" (OuterVolumeSpecName: "istio-data") pod "8352ba85-d596-4127-8ee5-c7922f885924" (UID: "8352ba85-d596-4127-8ee5-c7922f885924"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:11.917579 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.917548    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-istio-token" (OuterVolumeSpecName: "istio-token") pod "8352ba85-d596-4127-8ee5-c7922f885924" (UID: "8352ba85-d596-4127-8ee5-c7922f885924"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:02:11.917709 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.917651    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8352ba85-d596-4127-8ee5-c7922f885924" (UID: "8352ba85-d596-4127-8ee5-c7922f885924"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:11.917965 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.917945    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-kube-api-access-8hvnd" (OuterVolumeSpecName: "kube-api-access-8hvnd") pod "8352ba85-d596-4127-8ee5-c7922f885924" (UID: "8352ba85-d596-4127-8ee5-c7922f885924"). InnerVolumeSpecName "kube-api-access-8hvnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:02:11.918057 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:11.918001    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8352ba85-d596-4127-8ee5-c7922f885924-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8352ba85-d596-4127-8ee5-c7922f885924" (UID: "8352ba85-d596-4127-8ee5-c7922f885924"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:02:12.017049 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.017011    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-envoy\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:02:12.017049 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.017043    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hvnd\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-kube-api-access-8hvnd\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:02:12.017049 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.017054    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8352ba85-d596-4127-8ee5-c7922f885924-istiod-ca-cert\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:02:12.017291 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.017063    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8352ba85-d596-4127-8ee5-c7922f885924-istio-podinfo\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:02:12.017291 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.017072    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-certs\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:02:12.017291 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.017080    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-workload-socket\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:02:12.017291 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.017089    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8352ba85-d596-4127-8ee5-c7922f885924-istio-token\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:02:12.017291 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.017097    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8352ba85-d596-4127-8ee5-c7922f885924-istio-data\") on node \"ip-10-0-138-237.ec2.internal\" DevicePath \"\""
Jul 21 13:02:12.119198 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.119159    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"]
Jul 21 13:02:12.123052 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.123025    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-48842b-openshift-default-6f4f4b7f67-ls9xv"]
Jul 21 13:02:12.417338 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.417243    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8352ba85-d596-4127-8ee5-c7922f885924" path="/var/lib/kubelet/pods/8352ba85-d596-4127-8ee5-c7922f885924/volumes"
Jul 21 13:02:12.728056 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:12.727979    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7966fbf488-xd7nf_587525a2-6503-413c-9b40-fbe28c8b41c4/manager/0.log"
Jul 21 13:02:13.245834 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:13.245803    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58f85c6b8d-gmbzq_7bd1806e-5e0f-4baa-b3d5-35776ff78d94/copy-manifests/0.log"
Jul 21 13:02:13.260583 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:13.260552    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58f85c6b8d-gmbzq_7bd1806e-5e0f-4baa-b3d5-35776ff78d94/manager/0.log"
Jul 21 13:02:13.830626 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:13.830594    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743950-lsvbh_4b240fbd-962c-4c4c-b2f9-03dfb124b536/cleanup/0.log"
Jul 21 13:02:14.940019 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:14.939994    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh_3c45448a-d2cb-4b9e-8588-9c6324577b8b/util/0.log"
Jul 21 13:02:14.947079 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:14.947054    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh_3c45448a-d2cb-4b9e-8588-9c6324577b8b/pull/0.log"
Jul 21 13:02:14.953954 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:14.953934    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh_3c45448a-d2cb-4b9e-8588-9c6324577b8b/extract/0.log"
Jul 21 13:02:15.063448 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:15.063415    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5_06de834d-245a-430b-b7cd-32d73861b933/extract/0.log"
Jul 21 13:02:15.069859 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:15.069833    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5_06de834d-245a-430b-b7cd-32d73861b933/util/0.log"
Jul 21 13:02:15.076430 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:15.076408    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5_06de834d-245a-430b-b7cd-32d73861b933/pull/0.log"
Jul 21 13:02:15.196582 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:15.196486    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c_56f76b3b-58fe-402b-9b02-938436473fc7/extract/0.log"
Jul 21 13:02:15.202722 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:15.202702    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c_56f76b3b-58fe-402b-9b02-938436473fc7/util/0.log"
Jul 21 13:02:15.209014 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:15.208987    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c_56f76b3b-58fe-402b-9b02-938436473fc7/pull/0.log"
Jul 21 13:02:15.315290 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:15.315257    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh_789e13b0-0f43-4814-8311-95da61ce53f1/extract/0.log"
Jul 21 13:02:15.321143 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:15.321107    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh_789e13b0-0f43-4814-8311-95da61ce53f1/util/0.log"
Jul 21 13:02:15.327947 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:15.327917    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh_789e13b0-0f43-4814-8311-95da61ce53f1/pull/0.log"
Jul 21 13:02:16.179467 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:16.179434    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wjrqj_f3f4473a-baac-414d-9ccc-f561b6286305/limitador/0.log"
Jul 21 13:02:16.806135 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:16.806112    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pfllr_a86f1e20-097a-4546-8ee8-232bb2c7c776/discovery/0.log"
Jul 21 13:02:17.428391 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:17.428347    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hrhxm_b378e830-e542-4069-8f28-ff2ab7443874/payload-processing/0.log"
Jul 21 13:02:18.365686 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:18.365653    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-qbhs4_5cf0343f-6b3f-4d6e-b904-5dcaf13ef1b8/main/0.log"
Jul 21 13:02:27.532813 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:27.532752    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q7tnn_5b6c7079-8edc-469b-af1c-61f5ccfa0511/global-pull-secret-syncer/0.log"
Jul 21 13:02:27.812841 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:27.812812    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c2ltb_70679208-3538-46cc-8b71-0e27edb8344c/konnectivity-agent/0.log"
Jul 21 13:02:28.010130 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:28.010092    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-237.ec2.internal_378731304c8222449ffad4ab0f72e94a/haproxy/0.log"
Jul 21 13:02:32.323255 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.323228    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh_3c45448a-d2cb-4b9e-8588-9c6324577b8b/extract/0.log"
Jul 21 13:02:32.348919 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.348891    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh_3c45448a-d2cb-4b9e-8588-9c6324577b8b/util/0.log"
Jul 21 13:02:32.374645 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.374622    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wsxhh_3c45448a-d2cb-4b9e-8588-9c6324577b8b/pull/0.log"
Jul 21 13:02:32.409028 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.409000    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5_06de834d-245a-430b-b7cd-32d73861b933/extract/0.log"
Jul 21 13:02:32.438726 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.438698    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5_06de834d-245a-430b-b7cd-32d73861b933/util/0.log"
Jul 21 13:02:32.475701 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.475674    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j4cj5_06de834d-245a-430b-b7cd-32d73861b933/pull/0.log"
Jul 21 13:02:32.503971 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.503949    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c_56f76b3b-58fe-402b-9b02-938436473fc7/extract/0.log"
Jul 21 13:02:32.531733 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.531704    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c_56f76b3b-58fe-402b-9b02-938436473fc7/util/0.log"
Jul 21 13:02:32.563533 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.563507    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jgv2c_56f76b3b-58fe-402b-9b02-938436473fc7/pull/0.log"
Jul 21 13:02:32.593837 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.593747    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh_789e13b0-0f43-4814-8311-95da61ce53f1/extract/0.log"
Jul 21 13:02:32.618884 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.618856    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh_789e13b0-0f43-4814-8311-95da61ce53f1/util/0.log"
Jul 21 13:02:32.643584 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:32.643554    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1db2kh_789e13b0-0f43-4814-8311-95da61ce53f1/pull/0.log"
Jul 21 13:02:33.513536 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:33.513507    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wjrqj_f3f4473a-baac-414d-9ccc-f561b6286305/limitador/0.log"
Jul 21 13:02:35.343613 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.343584    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f1f12cb-ae83-409a-90ea-fa45b7f7434f/alertmanager/0.log"
Jul 21 13:02:35.369375 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.369344    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f1f12cb-ae83-409a-90ea-fa45b7f7434f/config-reloader/0.log"
Jul 21 13:02:35.391209 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.391185    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f1f12cb-ae83-409a-90ea-fa45b7f7434f/kube-rbac-proxy-web/0.log"
Jul 21 13:02:35.412944 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.412925    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f1f12cb-ae83-409a-90ea-fa45b7f7434f/kube-rbac-proxy/0.log"
Jul 21 13:02:35.437381 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.437357    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f1f12cb-ae83-409a-90ea-fa45b7f7434f/kube-rbac-proxy-metric/0.log"
Jul 21 13:02:35.465966 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.465938    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f1f12cb-ae83-409a-90ea-fa45b7f7434f/prom-label-proxy/0.log"
Jul 21 13:02:35.490521 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.490494    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1f1f12cb-ae83-409a-90ea-fa45b7f7434f/init-config-reloader/0.log"
Jul 21 13:02:35.792888 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.792807    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66whn_b9f9b59c-5d84-4796-9691-4248b1a94346/node-exporter/0.log"
Jul 21 13:02:35.813817 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.813785    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66whn_b9f9b59c-5d84-4796-9691-4248b1a94346/kube-rbac-proxy/0.log"
Jul 21 13:02:35.835184 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:35.835159    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66whn_b9f9b59c-5d84-4796-9691-4248b1a94346/init-textfile/0.log"
Jul 21 13:02:37.788476 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788439    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"]
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788728    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d91abf12-e61a-4c2a-803e-e1b0d304a469" containerName="maas-api"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788739    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d91abf12-e61a-4c2a-803e-e1b0d304a469" containerName="maas-api"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788751    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788757    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788764    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788784    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788802    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23fbd862-29a8-4376-9b30-9cab8122e30e" containerName="maas-api"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788811    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="23fbd862-29a8-4376-9b30-9cab8122e30e" containerName="maas-api"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788819    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788824    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788830    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788835    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788888    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8352ba85-d596-4127-8ee5-c7922f885924" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788894    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788900    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="23fbd862-29a8-4376-9b30-9cab8122e30e" containerName="maas-api"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788907    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d91abf12-e61a-4c2a-803e-e1b0d304a469" containerName="maas-api"
Jul 21 13:02:37.788911 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.788913    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d085d6ea-147e-4295-89d8-143eec1b224f" containerName="istio-proxy"
Jul 21 13:02:37.791963 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.791942    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.794427 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.794397    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqg59\"/\"openshift-service-ca.crt\""
Jul 21 13:02:37.794603 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.794581    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqg59\"/\"kube-root-ca.crt\""
Jul 21 13:02:37.795256 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.795236    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nqg59\"/\"default-dockercfg-6wm6t\""
Jul 21 13:02:37.805958 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.805933    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"]
Jul 21 13:02:37.841884 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.841837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-proc\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.841884 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.841879    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-sys\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.842091 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.841920    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-podres\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.842091 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.841951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4kzv\" (UniqueName: \"kubernetes.io/projected/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-kube-api-access-n4kzv\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.842091 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.841996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-lib-modules\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.943201 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.943166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-proc\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.943201 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.943204    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-sys\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.943466 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.943231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-podres\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.943466 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.943317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-sys\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.943466 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.943317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-proc\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.943466 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.943363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4kzv\" (UniqueName: \"kubernetes.io/projected/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-kube-api-access-n4kzv\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.943466 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.943403    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-lib-modules\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.943466 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.943415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-podres\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.943754 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.943478    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-lib-modules\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:37.951522 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:37.951497    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4kzv\" (UniqueName: \"kubernetes.io/projected/ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1-kube-api-access-n4kzv\") pod \"perf-node-gather-daemonset-ds8xj\" (UID: \"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:38.102561 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:38.102516    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:38.236230 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:38.236203    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"]
Jul 21 13:02:38.238808 ip-10-0-138-237 kubenswrapper[2562]: W0721 13:02:38.238755    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podee0ae0e1_a00c_4417_ba1a_828f51ad2cd1.slice/crio-c5accfdc828c21590ebbb64715c1a3fe5adc1d29fb12a713d6ff966135e3d574 WatchSource:0}: Error finding container c5accfdc828c21590ebbb64715c1a3fe5adc1d29fb12a713d6ff966135e3d574: Status 404 returned error can't find the container with id c5accfdc828c21590ebbb64715c1a3fe5adc1d29fb12a713d6ff966135e3d574
Jul 21 13:02:38.925396 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:38.925355    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj" event={"ID":"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1","Type":"ContainerStarted","Data":"aedd1fff760391fe801280cdd65b003bf1f9363427dda63e5af889071dda6148"}
Jul 21 13:02:38.925922 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:38.925436    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj" event={"ID":"ee0ae0e1-a00c-4417-ba1a-828f51ad2cd1","Type":"ContainerStarted","Data":"c5accfdc828c21590ebbb64715c1a3fe5adc1d29fb12a713d6ff966135e3d574"}
Jul 21 13:02:38.925922 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:38.925518    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:38.944057 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:38.944005    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj" podStartSLOduration=1.9439891660000002 podStartE2EDuration="1.943989166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:02:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:02:38.940963798 +0000 UTC m=+3082.999394839" watchObservedRunningTime="2026-07-21 13:02:38.943989166 +0000 UTC m=+3083.002420171"
Jul 21 13:02:40.788555 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:40.788526    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rwmgb_4b3185c6-d223-4671-a986-0858e36e5190/dns/0.log"
Jul 21 13:02:40.810646 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:40.810612    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rwmgb_4b3185c6-d223-4671-a986-0858e36e5190/kube-rbac-proxy/0.log"
Jul 21 13:02:40.835679 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:40.835650    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-75gck_163cac4c-f48f-41af-9e32-faf67720471d/dns-node-resolver/0.log"
Jul 21 13:02:41.538765 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:41.538729    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vfgrc_b848d323-22e3-44fa-a82b-2994f66c296a/node-ca/0.log"
Jul 21 13:02:42.760341 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:42.760308    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pfllr_a86f1e20-097a-4546-8ee8-232bb2c7c776/discovery/0.log"
Jul 21 13:02:42.956298 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:42.956258    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hrhxm_b378e830-e542-4069-8f28-ff2ab7443874/payload-processing/0.log"
Jul 21 13:02:43.777285 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:43.777251    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tb2sh_ce7c7b6d-ce2c-4026-83de-c59e7d3abae9/serve-healthcheck-canary/0.log"
Jul 21 13:02:44.410613 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:44.410580    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdb7w_7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee/kube-rbac-proxy/0.log"
Jul 21 13:02:44.431099 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:44.431068    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdb7w_7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee/exporter/0.log"
Jul 21 13:02:44.454412 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:44.454374    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdb7w_7b9466a7-98d1-467c-9ce7-5b69fdf5b0ee/extractor/0.log"
Jul 21 13:02:44.939948 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:44.939922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-ds8xj"
Jul 21 13:02:47.607195 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:47.607157    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7966fbf488-xd7nf_587525a2-6503-413c-9b40-fbe28c8b41c4/manager/0.log"
Jul 21 13:02:47.880123 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:47.880090    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58f85c6b8d-gmbzq_7bd1806e-5e0f-4baa-b3d5-35776ff78d94/manager/0.log"
Jul 21 13:02:47.904753 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:47.904722    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58f85c6b8d-gmbzq_7bd1806e-5e0f-4baa-b3d5-35776ff78d94/copy-manifests/0.log"
Jul 21 13:02:49.004622 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:49.004582    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pfllr_a86f1e20-097a-4546-8ee8-232bb2c7c776/discovery/0.log"
Jul 21 13:02:49.158299 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:49.158217    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hrhxm_b378e830-e542-4069-8f28-ff2ab7443874/payload-processing/0.log"
Jul 21 13:02:49.251972 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:49.251943    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c65cd9d66-jmsxk_21e68d0a-af7e-4183-82d7-c5b1843de9ac/manager/0.log"
Jul 21 13:02:50.648861 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:50.648813    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pfllr_a86f1e20-097a-4546-8ee8-232bb2c7c776/discovery/0.log"
Jul 21 13:02:50.792305 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:50.792273    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hrhxm_b378e830-e542-4069-8f28-ff2ab7443874/payload-processing/0.log"
Jul 21 13:02:54.293330 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:54.293296    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-k9qlv_c42d6ab4-ba5a-4b34-bd34-82652798ff36/migrator/0.log"
Jul 21 13:02:54.313094 ip-10-0-138-237 kubenswrapper[2562]: I0721 13:02:54.313067    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-k9qlv_c42d6ab4-ba5a-4b34-bd34-82652798ff36/graceful-termination/0.log"