Jul 27 21:31:40.211627 ip-10-0-132-161 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 27 21:31:40.211641 ip-10-0-132-161 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 27 21:31:40.211650 ip-10-0-132-161 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 27 21:31:40.211972 ip-10-0-132-161 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 27 21:31:50.460662 ip-10-0-132-161 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 27 21:31:50.460680 ip-10-0-132-161 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 82ebf8bf8f01451ead9d80f81503a6c6 --
Jul 27 21:33:48.998972 ip-10-0-132-161 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 21:33:49.349764 ip-10-0-132-161 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 21:33:49.349764 ip-10-0-132-161 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 21:33:49.349764 ip-10-0-132-161 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 21:33:49.349764 ip-10-0-132-161 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 21:33:49.349764 ip-10-0-132-161 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 21:33:49.350634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.350545    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 21:33:49.355560 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355546    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:33:49.355560 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355559    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355563    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355567    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355571    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355574    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355576    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355579    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355582    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355585    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355587    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355590    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355592    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355595    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355598    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355605    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355608    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355611    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355613    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355616    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355618    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:33:49.355627 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355621    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355623    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355626    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355628    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355631    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355633    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355636    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355639    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355641    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355644    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355646    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355649    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355651    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355654    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355657    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355659    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355661    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355664    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355666    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:33:49.356131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355669    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355673    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355676    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355680    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355682    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355685    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355689    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355691    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355694    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355697    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355699    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355702    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355705    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355707    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355710    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355713    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355716    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355718    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355721    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355723    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:33:49.356621 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355726    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355739    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355742    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355744    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355747    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355750    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355754    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355756    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355760    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355762    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355765    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355768    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355770    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355773    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355776    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355779    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355782    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355785    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355788    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:33:49.357130 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355793    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355796    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355798    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355801    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355803    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355806    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355808    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355811    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355816    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355819    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355822    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355824    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355827    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355829    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355832    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355834    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355837    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355839    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355842    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.355844    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:33:49.357582 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357081    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357089    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357092    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357095    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357098    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357103    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357106    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357109    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357112    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357115    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357118    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357121    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357123    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357127    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357129    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357132    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357134    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357137    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357140    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:33:49.358084 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357143    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357146    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357149    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357152    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357155    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357159    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357162    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357165    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357167    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357170    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357172    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357175    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357177    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357180    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357183    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357186    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357188    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357191    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357193    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357195    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:33:49.358551 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357198    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357200    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357203    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357205    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357209    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357213    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357216    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357219    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357221    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357224    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357227    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357229    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357232    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357234    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357237    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357239    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357242    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357244    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357247    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:33:49.359050 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357249    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357252    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357254    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357257    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357259    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357262    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357264    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357267    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357269    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357272    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357275    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357277    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357280    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357282    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357284    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357287    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357289    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357292    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357294    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357296    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:33:49.359516 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357299    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357302    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357304    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357307    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357310    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357312    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357315    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357317    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357320    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357322    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357324    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357327    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357329    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357332    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357335    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357337    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357339    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357342    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357344    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357346    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:33:49.360120 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357349    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357682    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357687    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357689    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357692    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357695    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357697    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357700    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357702    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357706    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357709    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357711    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357713    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357716    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357719    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357721    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357724    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357726    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357745    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357748    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357751    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:33:49.360622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357753    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357756    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357758    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357761    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357763    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357766    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357769    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357771    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357774    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357776    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357779    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357781    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357783    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357786    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357789    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357791    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357794    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357796    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357798    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357801    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:33:49.361159 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357804    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357806    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357808    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357811    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357813    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357817    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357819    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357821    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357824    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357827    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357829    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357834    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357837    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357840    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357843    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357846    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357848    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357851    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357853    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:33:49.361652 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357856    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357859    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357862    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357865    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357867    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357869    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357872    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357874    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357876    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357879    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357881    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357883    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357886    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357888    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357891    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357893    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357896    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357898    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357900    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:33:49.362215 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357904    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357906    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357909    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357912    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357914    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357916    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357919    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357921    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357924    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357927    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357929    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357932    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357934    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357937    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357939    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357942    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357944    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357947    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357949    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:33:49.362677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357953    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.357956    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358023    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358030    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358036    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358040    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358044    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358046    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358050    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358054    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358057    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358060    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358063    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358066    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358069    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358072    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358075    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358078    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358081    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358084    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358087    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358090    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358094    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358096    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 21:33:49.363203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358100    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358103    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358106    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358108    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358112    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358114    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358117    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358123    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358126    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358129    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358133    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358136    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358138    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358141    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358144    2578 flags.go:64] FLAG: --enable-server="true"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358147    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358151    2578 flags.go:64] FLAG: --event-burst="100"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358153    2578 flags.go:64] FLAG: --event-qps="50"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358156    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358159    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358162    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358165    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358168    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358171    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358174    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 27 21:33:49.363901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358177    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358179    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358182    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358185    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358187    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358190    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358193    2578 flags.go:64] FLAG: --feature-gates=""
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358196    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358199    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358201    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358204    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358207    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358210    2578 flags.go:64] FLAG: --help="false"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358212    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358217    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358220    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358223    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358226    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358229    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358232    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358234    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358237    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358240    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 21:33:49.364521 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358242    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358245    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358248    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358251    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358253    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358256    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358259    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358263    2578 flags.go:64] FLAG: --lock-file=""
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358266    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358269    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358272    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358277    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358280    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358282    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358285    2578 flags.go:64] FLAG: --logging-format="text"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358288    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358291    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358294    2578 flags.go:64] FLAG: --manifest-url=""
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358296    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358300    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358303    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358307    2578 flags.go:64] FLAG: --max-pods="110"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358310    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358313    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358316    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 21:33:49.365122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358319    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358322    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358324    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358327    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358333    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358336    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358339    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358342    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358345    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358349    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358352    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358355    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358358    2578 flags.go:64] FLAG: --port="10250"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358361    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358364    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0439d45ef4df50947"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358367    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358371    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358374    2578 flags.go:64] FLAG: --register-node="true"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358377    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358380    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358383    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358386    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358389    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358397    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358400    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 21:33:49.365726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358403    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358405    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358408    2578 flags.go:64] FLAG: --runonce="false"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358411    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358414    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358418    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358421    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358424    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358427    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358430    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358432    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358435    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358438    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358440    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358443    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358446    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358449    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358452    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358457    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358459    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358462    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358465    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358470    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358473    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358478    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 21:33:49.366341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358480    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358483    2578 flags.go:64] FLAG: --v="2"
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358487    2578 flags.go:64] FLAG: --version="false"
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358490    2578 flags.go:64] FLAG: --vmodule=""
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358494    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.358497    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358611    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358616    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358619    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358622    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358625    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358627    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358632    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358634    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358637    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358639    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358642    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358644    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358647    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358649    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:33:49.366979 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358652    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358654    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358657    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358659    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358662    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358665    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358667    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358670    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358672    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358677    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358680    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358682    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358690    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358693    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358696    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358698    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358701    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358703    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358705    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358708    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:33:49.367470 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358711    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358713    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358715    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358718    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358721    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358724    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358727    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358744    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358747    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358749    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358752    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358755    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358758    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358760    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358763    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358765    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358768    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358770    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358773    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358775    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:33:49.367980 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358778    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358781    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358784    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358786    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358789    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358792    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358795    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358797    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358800    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358802    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358804    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358807    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358810    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358812    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358815    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358818    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358821    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358824    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358826    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:33:49.368472 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358829    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358832    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358836    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358840    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358843    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358846    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358849    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358851    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358854    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358856    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358859    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358861    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358864    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358866    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358871    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358874    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358877    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358879    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358882    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:33:49.369030 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358889    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358892    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358894    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358897    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358899    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358902    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.358904    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.359519    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.365750    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.365764    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.366496    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.366507    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366560    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366565    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366568    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:33:49.369483 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366571    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366574    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366576    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366579    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366582    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366584    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366587    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366590    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366592    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366595    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366597    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366600    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366602    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366607    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366611    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366613    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366616    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366619    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366621    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366624    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:33:49.369874 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366626    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366629    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366631    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366634    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366636    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366639    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366642    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366645    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366647    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366649    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366652    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366654    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366657    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366659    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366663    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366666    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366669    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366672    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366675    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366678    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:33:49.370356 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366681    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366684    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366686    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366689    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366691    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366695    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366697    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366700    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366703    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366705    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366708    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366710    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366712    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366715    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366717    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366720    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366723    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366726    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366740    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366744    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:33:49.370859 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366746    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366749    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366752    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366754    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366756    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366759    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366762    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366764    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366767    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366769    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366772    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366774    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366777    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366779    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366781    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366784    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366786    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366789    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366792    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366794    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:33:49.371344 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366797    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366799    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366802    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366805    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366807    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366810    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366814    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366818    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366821    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366824    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366826    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366829    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366832    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366834    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366837    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:33:49.371844 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366839    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.366844    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366953    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366958    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366960    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366963    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366966    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366968    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366971    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366973    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366976    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366980    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366984    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366987    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366990    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:33:49.372208 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366993    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366996    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.366999    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367002    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367004    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367007    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367009    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367011    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367014    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367016    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367019    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367022    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367024    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367027    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367029    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367032    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367035    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367037    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367040    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367042    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:33:49.372584 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367044    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367047    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367050    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367052    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367054    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367057    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367059    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367061    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367064    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367066    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367069    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367071    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367074    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367076    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367078    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367081    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367083    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367085    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367088    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:33:49.373124 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367090    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367094    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367097    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367100    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367103    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367105    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367108    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367111    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367113    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367116    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367119    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367121    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367125    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367128    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367130    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367133    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367135    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367137    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367140    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:33:49.373568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367142    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367145    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367147    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367149    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367152    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367154    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367156    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367159    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367161    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367163    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367166    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367168    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367170    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367173    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367175    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367177    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367180    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367182    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367184    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367187    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:33:49.374053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367189    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367192    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367194    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367197    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367199    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367202    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367204    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:49.367207    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.367211    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.367307    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.369080    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.370267    2578 server.go:1019] "Starting client certificate rotation"
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.370368    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 21:33:49.374549 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.370410    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 21:33:49.374875 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.372589    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 21:33:49.378345 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.378326    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 21:33:49.388912 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.388894    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 27 21:33:49.390640 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.390625    2578 log.go:25] "Validated CRI v1 image API"
Jul 27 21:33:49.391345 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.391329    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 21:33:49.393342 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.393316    2578 fs.go:135] Filesystem UUIDs: map[5f49d428-870e-4de2-a667-19eae53df050:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 dee833af-a792-4dce-9ca2-e329cce559c2:/dev/nvme0n1p4]
Jul 27 21:33:49.393432 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.393340    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 21:33:49.397749 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.397713    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 21:33:49.398449 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.398343    2578 manager.go:216] Machine: {Timestamp:2026-07-27 21:33:49.397014632 +0000 UTC m=+0.292824074 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097662 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2af497c6bea42b2355c04cc8f180c8 SystemUUID:ec2af497-c6be-a42b-2355-c04cc8f180c8 BootID:82ebf8bf-8f01-451e-ad9d-80f81503a6c6 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:88:23:91:4d:c9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:88:23:91:4d:c9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:fb:4b:3d:ab:c3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 21:33:49.398449 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.398445    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 21:33:49.398590 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.398536    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 21:33:49.401802 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.401781    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 21:33:49.401932 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.401805    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-161.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 21:33:49.403152 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.403141    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 21:33:49.403184 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.403156    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 21:33:49.403184 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.403163    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 21:33:49.403798 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.403788    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 21:33:49.405258 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.405248    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 21:33:49.407654 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.407643    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 21:33:49.407824 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.407814    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 21:33:49.407862 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.407846    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 21:33:49.407862 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.407858    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 21:33:49.407916 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.407869    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 21:33:49.407916 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.407878    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 27 21:33:49.407916 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.407886    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 21:33:49.408214 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.408198    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 21:33:49.408259 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.408221    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 21:33:49.410429 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.410415    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 21:33:49.412036 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.412023    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 21:33:49.412103 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.412053    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 21:33:49.413663 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413652    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 21:33:49.413703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413668    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 21:33:49.413703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413674    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 21:33:49.413703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413679    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 21:33:49.413703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413686    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 21:33:49.413703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413691    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 21:33:49.413703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413699    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 21:33:49.413703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413704    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 21:33:49.413907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413710    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 21:33:49.413907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413715    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 21:33:49.413907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413724    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 21:33:49.413907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.413744    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 21:33:49.414266 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.414256    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 21:33:49.414266 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.414264    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 21:33:49.417262 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.417249    2578 server.go:1268] "Started kubelet"
Jul 27 21:33:49.417495 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.417421    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 21:33:49.420842 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.417455    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 21:33:49.421309 ip-10-0-132-161 systemd[1]: Started Kubernetes Kubelet.
Jul 27 21:33:49.422414 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.422122    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 21:33:49.423283 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.423261    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 21:33:49.423709 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.423683    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 21:33:49.424424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.424406    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 21:33:49.425683 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.425655    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 21:33:49.425683 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.425670    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 21:33:49.425683 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.425688    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-161.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 21:33:49.426433 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.425648    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-161.ec2.internal.18c6430deeb1728d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-161.ec2.internal,UID:ip-10-0-132-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-161.ec2.internal,},FirstTimestamp:2026-07-27 21:33:49.417222797 +0000 UTC m=+0.313032239,LastTimestamp:2026-07-27 21:33:49.417222797 +0000 UTC m=+0.313032239,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-161.ec2.internal,}"
Jul 27 21:33:49.429615 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.429601    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 21:33:49.429672 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.429615    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 21:33:49.429715 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.429700    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:49.430526 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.430071    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 21:33:49.430526 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.430085    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 21:33:49.430526 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.430098    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 21:33:49.430526 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.430225    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 21:33:49.430526 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.430237    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 21:33:49.432046 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.432017    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 21:33:49.432143 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.432051    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 21:33:49.432366 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.432354    2578 factory.go:153] Registering CRI-O factory
Jul 27 21:33:49.432430 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.432387    2578 factory.go:223] Registration of the crio container factory successfully
Jul 27 21:33:49.432430 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.432425    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 21:33:49.432430 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.432431    2578 factory.go:55] Registering systemd factory
Jul 27 21:33:49.432578 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.432437    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 27 21:33:49.432578 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.432457    2578 factory.go:103] Registering Raw factory
Jul 27 21:33:49.432578 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.432466    2578 manager.go:1196] Started watching for new ooms in manager
Jul 27 21:33:49.432940 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.432927    2578 manager.go:349] Starting recovery of all containers
Jul 27 21:33:49.442716 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.442589    2578 manager.go:354] Recovery completed
Jul 27 21:33:49.446579 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.446568    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:33:49.447622 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.447603    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l6fgr"
Jul 27 21:33:49.448175 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.448160    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:33:49.448248 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.448191    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:33:49.448248 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.448206    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:33:49.448674 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.448660    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 21:33:49.448674 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.448672    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:33:49.448838 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.448673    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 21:33:49.448838 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.448754    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 21:33:49.449687 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.449672    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:33:49.449772 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.449698    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:33:49.449772 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.449707    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:33:49.450409 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.450398    2578 policy_none.go:49] "None policy: Start"
Jul 27 21:33:49.450445 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.450415    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 21:33:49.450445 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.450425    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 21:33:49.451308 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.451298    2578 policy_none.go:47] "Start"
Jul 27 21:33:49.456594 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.456575    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l6fgr"
Jul 27 21:33:49.458716 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.458652    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-161.ec2.internal.18c6430df089ba9e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-161.ec2.internal,UID:ip-10-0-132-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-161.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-161.ec2.internal,},FirstTimestamp:2026-07-27 21:33:49.448174238 +0000 UTC m=+0.343983684,LastTimestamp:2026-07-27 21:33:49.448174238 +0000 UTC m=+0.343983684,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-161.ec2.internal,}"
Jul 27 21:33:49.491254 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.491237    2578 manager.go:337] "Starting Device Plugin manager"
Jul 27 21:33:49.491323 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.491266    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 21:33:49.491323 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.491275    2578 server.go:87] "Starting device plugin registration server"
Jul 27 21:33:49.491500 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.491487    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 21:33:49.491561 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.491501    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 21:33:49.491683 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.491661    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 21:33:49.491800 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.491760    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 21:33:49.491800 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.491770    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 21:33:49.492175 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.492151    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 21:33:49.492222 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.492198    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:49.592177 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.592131    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:33:49.593162 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.592933    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:33:49.593162 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.592956    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:33:49.593162 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.592967    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:33:49.593162 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.592994    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.597766 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.597747    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 21:33:49.598943 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.598926    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 21:33:49.599022 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.598965    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 21:33:49.599022 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.598982    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 21:33:49.599022 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.599008    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 21:33:49.601005 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.600960    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:33:49.601872 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.601851    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.601872 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.601871    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-161.ec2.internal\": node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:49.637492 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.637474    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:49.699373 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.699353    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal"]
Jul 27 21:33:49.699445 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.699416    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:33:49.701048 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.701033    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:33:49.701125 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.701064    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:33:49.701125 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.701073    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:33:49.702776 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.702764    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:33:49.702888 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.702873    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.702924 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.702900    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:33:49.703533 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.703516    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:33:49.703613 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.703548    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:33:49.703613 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.703563    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:33:49.703613 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.703522    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:33:49.703703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.703625    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:33:49.703703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.703635    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:33:49.704958 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.704945    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.704998 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.704977    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:33:49.705703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.705689    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:33:49.705769 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.705715    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:33:49.705769 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.705725    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:33:49.727272 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.727252    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-161.ec2.internal\" not found" node="ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.731508 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.731491    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-161.ec2.internal\" not found" node="ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.732536 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.732518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0c7066e91c3e4c6ab197571918f76a0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal\" (UID: \"0c7066e91c3e4c6ab197571918f76a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.732584 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.732545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c7066e91c3e4c6ab197571918f76a0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal\" (UID: \"0c7066e91c3e4c6ab197571918f76a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.732584 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.732561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19538ed6d308dc3f353d7b9a9b7dce11-config\") pod \"kube-apiserver-proxy-ip-10-0-132-161.ec2.internal\" (UID: \"19538ed6d308dc3f353d7b9a9b7dce11\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.737559 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.737545    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:49.832918 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.832899    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0c7066e91c3e4c6ab197571918f76a0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal\" (UID: \"0c7066e91c3e4c6ab197571918f76a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.832982 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.832925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c7066e91c3e4c6ab197571918f76a0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal\" (UID: \"0c7066e91c3e4c6ab197571918f76a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.832982 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.832943    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19538ed6d308dc3f353d7b9a9b7dce11-config\") pod \"kube-apiserver-proxy-ip-10-0-132-161.ec2.internal\" (UID: \"19538ed6d308dc3f353d7b9a9b7dce11\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.833044 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.833003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0c7066e91c3e4c6ab197571918f76a0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal\" (UID: \"0c7066e91c3e4c6ab197571918f76a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.833072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.833003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c7066e91c3e4c6ab197571918f76a0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal\" (UID: \"0c7066e91c3e4c6ab197571918f76a0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.833072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:49.833009    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19538ed6d308dc3f353d7b9a9b7dce11-config\") pod \"kube-apiserver-proxy-ip-10-0-132-161.ec2.internal\" (UID: \"19538ed6d308dc3f353d7b9a9b7dce11\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:49.838017 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.837996    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:49.938796 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:49.938748    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:50.028965 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.028940    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:50.034867 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.034846    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:50.039325 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:50.039307    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:50.139878 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:50.139842    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:50.240453 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:50.240396    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:50.340874 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:50.340851    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:50.370009 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.369992    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 21:33:50.370356 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.370115    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 21:33:50.438146 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.438127    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:33:50.440967 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:50.440952    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:50.461520 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.461492    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 21:28:50 +0000 UTC" deadline="2027-12-27 22:12:09.42444422 +0000 UTC"
Jul 27 21:33:50.461573 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.461517    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12432h38m18.962929817s"
Jul 27 21:33:50.500284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.500242    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:33:50.523798 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.523780    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:33:50.526457 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.526437    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:33:50.526557 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.526470    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:33:50.526557 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.526486    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:33:50.541151 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:50.541132    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-161.ec2.internal\" not found"
Jul 27 21:33:50.578962 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.578943    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:33:50.609573 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:50.609551    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19538ed6d308dc3f353d7b9a9b7dce11.slice/crio-abd40f322c29186063960ec5d796c1830bc2b5a7d01f5d2f1a56a2568065a3a2 WatchSource:0}: Error finding container abd40f322c29186063960ec5d796c1830bc2b5a7d01f5d2f1a56a2568065a3a2: Status 404 returned error can't find the container with id abd40f322c29186063960ec5d796c1830bc2b5a7d01f5d2f1a56a2568065a3a2
Jul 27 21:33:50.613658 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.613645    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:33:50.630582 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.630562    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:50.643541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.643526    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 21:33:50.644359 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.644346    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal"
Jul 27 21:33:50.653086 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:50.653072    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 21:33:50.668114 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:50.668092    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c7066e91c3e4c6ab197571918f76a0f.slice/crio-9b1f163008503c7053f4f860acd0b7d5549214dad9ea27638d08faebd99186a9 WatchSource:0}: Error finding container 9b1f163008503c7053f4f860acd0b7d5549214dad9ea27638d08faebd99186a9: Status 404 returned error can't find the container with id 9b1f163008503c7053f4f860acd0b7d5549214dad9ea27638d08faebd99186a9
Jul 27 21:33:51.408845 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.408817    2578 apiserver.go:52] "Watching apiserver"
Jul 27 21:33:51.417669 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.417643    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 21:33:51.419496 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.419464    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p","openshift-image-registry/node-ca-22g6x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal","openshift-multus/multus-qgbwm","openshift-multus/network-metrics-daemon-wtmrw","openshift-network-diagnostics/network-check-target-4g95m","openshift-network-operator/iptables-alerter-xdmr5","kube-system/global-pull-secret-syncer-2q7dv","kube-system/konnectivity-agent-q5jzr","openshift-cluster-node-tuning-operator/tuned-2lb6z","openshift-dns/node-resolver-ftzbq","openshift-multus/multus-additional-cni-plugins-xtpwr","openshift-ovn-kubernetes/ovnkube-node-g4p2b"]
Jul 27 21:33:51.421318 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.421288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:33:51.423019 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.422998    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.424347 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.424324    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 21:33:51.424482 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.424450    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 21:33:51.424562 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.424543    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zv6n8\""
Jul 27 21:33:51.424788 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.424761    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.425843 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.425821    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.426589 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.426568    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 21:33:51.426678 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.426597    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fsgcq\""
Jul 27 21:33:51.426678 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.426637    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 21:33:51.426827 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.426796    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 21:33:51.427047 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.427028    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 21:33:51.427115 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.427031    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 21:33:51.427471 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.427450    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 21:33:51.427787 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.427764    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 21:33:51.427924 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.427908    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m5fqm\""
Jul 27 21:33:51.428021 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.427996    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 21:33:51.428278 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.428260    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.428912 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.428890    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:51.429017 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.428958    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:33:51.429334 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.429311    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 21:33:51.429801 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.429781    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 21:33:51.430054 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.430034    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 21:33:51.430320 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.430302    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 21:33:51.432307 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.432280    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 21:33:51.432405 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.432324    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 21:33:51.432759 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.432742    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 21:33:51.433381 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.433359    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g5hgq\""
Jul 27 21:33:51.433546 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.433528    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 21:33:51.433947 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.433932    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bpkp7\""
Jul 27 21:33:51.434454 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.434417    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:51.434545 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.434527    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.434631 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.434546    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:33:51.437463 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.437431    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:51.437625 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.437600    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:33:51.438620 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.438141    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 21:33:51.438620 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.438151    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 21:33:51.438620 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.438161    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x6q7n\""
Jul 27 21:33:51.438620 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.438287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.438859 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.438673    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 21:33:51.441086 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441063    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.441188 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d32a5315-4ac2-4d35-b133-854854da98c7-host\") pod \"node-ca-22g6x\" (UID: \"d32a5315-4ac2-4d35-b133-854854da98c7\") " pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.441188 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-log-socket\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.441288 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441200    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-cni-bin\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.441288 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd3d4868-8036-4b10-b7ce-15d43f5851e1-iptables-alerter-script\") pod \"iptables-alerter-xdmr5\" (UID: \"bd3d4868-8036-4b10-b7ce-15d43f5851e1\") " pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.441288 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441262    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-cnibin\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.441446 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441287    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.441446 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-conf-dir\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.441446 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-var-lib-openvswitch\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.441446 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-run-ovn\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.441446 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-dbus\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:51.441446 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4057d2d1-dd7e-46e1-af61-4a26a094ca35-cni-binary-copy\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-etc-openvswitch\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-kubelet-config\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw4f6\" (UniqueName: \"kubernetes.io/projected/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-kube-api-access-rw4f6\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441588    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-var-lib-cni-multus\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-run-openvswitch\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-env-overrides\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-system-cni-dir\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441695    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-system-cni-dir\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-var-lib-kubelet\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.441742 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441743    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-sys-fs\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk6px\" (UniqueName: \"kubernetes.io/projected/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-kube-api-access-wk6px\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441850    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-cni-netd\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441901    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-cnibin\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-var-lib-cni-bin\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-registration-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441986    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-etc-selinux\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.441999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-slash\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-run-systemd\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442046    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-os-release\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-systemd-units\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.442239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/24413672-bc61-4434-aa07-40013e6257a6-konnectivity-ca\") pod \"konnectivity-agent-q5jzr\" (UID: \"24413672-bc61-4434-aa07-40013e6257a6\") " pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-run-k8s-cni-cncf-io\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442513    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-hostroot\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442541    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkx8s\" (UniqueName: \"kubernetes.io/projected/4ea190da-d6af-4616-93a3-4ad59b61282a-kube-api-access-vkx8s\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442573    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-ovnkube-config\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442621    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-ovn-node-metrics-cert\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd3d4868-8036-4b10-b7ce-15d43f5851e1-host-slash\") pod \"iptables-alerter-xdmr5\" (UID: \"bd3d4868-8036-4b10-b7ce-15d43f5851e1\") " pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442686    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442695    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-run-netns\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442701    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-daemon-config\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-run-ovn-kubernetes\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.442907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d32a5315-4ac2-4d35-b133-854854da98c7-serviceca\") pod \"node-ca-22g6x\" (UID: \"d32a5315-4ac2-4d35-b133-854854da98c7\") " pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442932    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-cni-dir\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.442959    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-os-release\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-kubelet\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/24413672-bc61-4434-aa07-40013e6257a6-agent-certs\") pod \"konnectivity-agent-q5jzr\" (UID: \"24413672-bc61-4434-aa07-40013e6257a6\") " pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-etc-kubernetes\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443141    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443038    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443325    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-26t5b\""
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443141    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443390    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-device-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptcnz\" (UniqueName: \"kubernetes.io/projected/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-kube-api-access-ptcnz\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-ovnkube-script-lib\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443488    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:33:51.443488 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-run-netns\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443520    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znl9f\" (UniqueName: \"kubernetes.io/projected/d32a5315-4ac2-4d35-b133-854854da98c7-kube-api-access-znl9f\") pod \"node-ca-22g6x\" (UID: \"d32a5315-4ac2-4d35-b133-854854da98c7\") " pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf62g\" (UniqueName: \"kubernetes.io/projected/4057d2d1-dd7e-46e1-af61-4a26a094ca35-kube-api-access-nf62g\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443590    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pr5j\" (UniqueName: \"kubernetes.io/projected/bd3d4868-8036-4b10-b7ce-15d43f5851e1-kube-api-access-8pr5j\") pod \"iptables-alerter-xdmr5\" (UID: \"bd3d4868-8036-4b10-b7ce-15d43f5851e1\") " pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-socket-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443690    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-run-multus-certs\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-node-log\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443741    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443766    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-socket-dir-parent\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.443806    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8c4jf\""
Jul 27 21:33:51.444468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.444129    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 21:33:51.445643 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.445496    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bzc5h\""
Jul 27 21:33:51.446017 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.445988    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 21:33:51.446378 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.446357    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 21:33:51.453377 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.453355    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 21:33:51.480362 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.480344    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bcj84"
Jul 27 21:33:51.490827 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.490801    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bcj84"
Jul 27 21:33:51.531282 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.531261    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 21:33:51.544239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.544343 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-run-multus-certs\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.544343 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-node-log\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.544343 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.544343 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-socket-dir-parent\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.544564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544366    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-sysctl-d\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.544564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d32a5315-4ac2-4d35-b133-854854da98c7-host\") pod \"node-ca-22g6x\" (UID: \"d32a5315-4ac2-4d35-b133-854854da98c7\") " pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.544564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-log-socket\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.544564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-cni-bin\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.544564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd3d4868-8036-4b10-b7ce-15d43f5851e1-iptables-alerter-script\") pod \"iptables-alerter-xdmr5\" (UID: \"bd3d4868-8036-4b10-b7ce-15d43f5851e1\") " pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.544564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-lib-modules\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.544564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544515    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-cnibin\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.544564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544586    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-conf-dir\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-var-lib-openvswitch\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-run-ovn\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-dbus\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-modprobe-d\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4057d2d1-dd7e-46e1-af61-4a26a094ca35-cni-binary-copy\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-etc-openvswitch\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-kubelet-config\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b29ce2d-9385-4403-8d0f-554572628d9a-tmp\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw4f6\" (UniqueName: \"kubernetes.io/projected/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-kube-api-access-rw4f6\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-var-lib-cni-multus\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544882    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-run-openvswitch\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544906    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-env-overrides\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544932    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-system-cni-dir\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544957    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-system-cni-dir\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.544987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.544980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-var-lib-kubelet\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545012    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-sys-fs\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk6px\" (UniqueName: \"kubernetes.io/projected/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-kube-api-access-wk6px\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545067    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545099    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f9f3fc-17d0-4b1d-8a50-01757a4f934c-tmp-dir\") pod \"node-resolver-ftzbq\" (UID: \"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c\") " pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-cni-netd\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-cnibin\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545222    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-var-lib-cni-bin\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545255    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-registration-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-etc-selinux\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545337    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-slash\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-run-systemd\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-os-release\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-systemd-units\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-systemd\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-host\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.545849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/24413672-bc61-4434-aa07-40013e6257a6-konnectivity-ca\") pod \"konnectivity-agent-q5jzr\" (UID: \"24413672-bc61-4434-aa07-40013e6257a6\") " pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-run-k8s-cni-cncf-io\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-hostroot\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545588    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkx8s\" (UniqueName: \"kubernetes.io/projected/4ea190da-d6af-4616-93a3-4ad59b61282a-kube-api-access-vkx8s\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545612    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-ovnkube-config\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-ovn-node-metrics-cert\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545667    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd3d4868-8036-4b10-b7ce-15d43f5851e1-host-slash\") pod \"iptables-alerter-xdmr5\" (UID: \"bd3d4868-8036-4b10-b7ce-15d43f5851e1\") " pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-kubernetes\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-run-netns\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-daemon-config\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-run-ovn-kubernetes\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545861    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-tuned\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tznl\" (UniqueName: \"kubernetes.io/projected/c9f9f3fc-17d0-4b1d-8a50-01757a4f934c-kube-api-access-8tznl\") pod \"node-resolver-ftzbq\" (UID: \"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c\") " pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545926    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d32a5315-4ac2-4d35-b133-854854da98c7-serviceca\") pod \"node-ca-22g6x\" (UID: \"d32a5315-4ac2-4d35-b133-854854da98c7\") " pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545952    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-cni-dir\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.545977    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-os-release\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-kubelet\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.546444 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-run\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546087    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-var-lib-kubelet\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/24413672-bc61-4434-aa07-40013e6257a6-agent-certs\") pod \"konnectivity-agent-q5jzr\" (UID: \"24413672-bc61-4434-aa07-40013e6257a6\") " pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-etc-kubernetes\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546179    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-device-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptcnz\" (UniqueName: \"kubernetes.io/projected/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-kube-api-access-ptcnz\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546337    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-ovnkube-script-lib\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-run-netns\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72lrs\" (UniqueName: \"kubernetes.io/projected/2b29ce2d-9385-4403-8d0f-554572628d9a-kube-api-access-72lrs\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546458    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znl9f\" (UniqueName: \"kubernetes.io/projected/d32a5315-4ac2-4d35-b133-854854da98c7-kube-api-access-znl9f\") pod \"node-ca-22g6x\" (UID: \"d32a5315-4ac2-4d35-b133-854854da98c7\") " pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546484    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf62g\" (UniqueName: \"kubernetes.io/projected/4057d2d1-dd7e-46e1-af61-4a26a094ca35-kube-api-access-nf62g\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pr5j\" (UniqueName: \"kubernetes.io/projected/bd3d4868-8036-4b10-b7ce-15d43f5851e1-kube-api-access-8pr5j\") pod \"iptables-alerter-xdmr5\" (UID: \"bd3d4868-8036-4b10-b7ce-15d43f5851e1\") " pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-sys\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.547045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-socket-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.547622 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546601    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-sysconfig\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.547622 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-sysctl-conf\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.547622 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9f9f3fc-17d0-4b1d-8a50-01757a4f934c-hosts-file\") pod \"node-resolver-ftzbq\" (UID: \"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c\") " pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.547622 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546915    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.547622 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.546965    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-run-multus-certs\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.547622 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.547011    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-node-log\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.547622 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.547592    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.547858 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.547688    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-socket-dir-parent\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.547858 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.547735    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d32a5315-4ac2-4d35-b133-854854da98c7-host\") pod \"node-ca-22g6x\" (UID: \"d32a5315-4ac2-4d35-b133-854854da98c7\") " pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.547858 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.547781    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-log-socket\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.547858 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.547848    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-cni-bin\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.548359 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.548329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd3d4868-8036-4b10-b7ce-15d43f5851e1-iptables-alerter-script\") pod \"iptables-alerter-xdmr5\" (UID: \"bd3d4868-8036-4b10-b7ce-15d43f5851e1\") " pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.548441 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.548405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-cnibin\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.548958 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.548878    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.549054 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.548976    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-conf-dir\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.549054 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.549017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-var-lib-openvswitch\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.549154 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.549061    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-run-ovn\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.549248 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.549204    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-dbus\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:51.549951 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.549828    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4057d2d1-dd7e-46e1-af61-4a26a094ca35-cni-binary-copy\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.549951 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.549922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-etc-openvswitch\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.550095 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.549976    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-kubelet-config\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:51.550300 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.550278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-var-lib-cni-multus\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.550371 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.550340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-run-netns\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.550427 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.550381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-run-openvswitch\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.550772 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.550738    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-env-overrides\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.550977 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.550848    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-system-cni-dir\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.550977 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.550937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-daemon-config\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.550977 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.550956    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-system-cni-dir\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.551226 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551022    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-var-lib-kubelet\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.551226 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.551226 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-sys-fs\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.551472 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-run-ovn-kubernetes\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.551472 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.551407    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:51.551579 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.551493    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret podName:d3b83b76-20e7-43e7-85bd-885ab2b22ce8 nodeName:}" failed. No retries permitted until 2026-07-27 21:33:52.051464111 +0000 UTC m=+2.947273560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret") pod "global-pull-secret-syncer-2q7dv" (UID: "d3b83b76-20e7-43e7-85bd-885ab2b22ce8") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:51.551686 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551657    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-cni-netd\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.551773 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551683    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d32a5315-4ac2-4d35-b133-854854da98c7-serviceca\") pod \"node-ca-22g6x\" (UID: \"d32a5315-4ac2-4d35-b133-854854da98c7\") " pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.551773 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551703    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-ovnkube-script-lib\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.551889 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551774    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-kubelet\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.551889 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551782    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-multus-cni-dir\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.551889 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551850    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-run-netns\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.552056 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.552634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.552147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.552634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.552231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-var-lib-cni-bin\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.552634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.552284    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-registration-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.552634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.552385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-etc-selinux\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.552634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.552413    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-host-slash\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.552634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.552437    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-run-systemd\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.552634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.552496    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-os-release\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.552634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.552541    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-systemd-units\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.553116 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.552829    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 21:33:51.553189 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.553161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/24413672-bc61-4434-aa07-40013e6257a6-konnectivity-ca\") pod \"konnectivity-agent-q5jzr\" (UID: \"24413672-bc61-4434-aa07-40013e6257a6\") " pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:33:51.553284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.553209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-socket-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.553284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.553217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-cnibin\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.553284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.551042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-device-dir\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.553284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.553232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-host-run-k8s-cni-cncf-io\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.553496 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.553342    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-os-release\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.553496 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.553455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd3d4868-8036-4b10-b7ce-15d43f5851e1-host-slash\") pod \"iptables-alerter-xdmr5\" (UID: \"bd3d4868-8036-4b10-b7ce-15d43f5851e1\") " pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.553596 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.553497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-hostroot\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.553596 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.553534    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:51.553596 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.553592    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs podName:4ea190da-d6af-4616-93a3-4ad59b61282a nodeName:}" failed. No retries permitted until 2026-07-27 21:33:52.053573878 +0000 UTC m=+2.949383324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs") pod "network-metrics-daemon-wtmrw" (UID: "4ea190da-d6af-4616-93a3-4ad59b61282a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:51.553782 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.553621    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4057d2d1-dd7e-46e1-af61-4a26a094ca35-etc-kubernetes\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.553782 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.553722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-ovnkube-config\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.557827 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.557757    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-ovn-node-metrics-cert\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.559366 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.559346    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/24413672-bc61-4434-aa07-40013e6257a6-agent-certs\") pod \"konnectivity-agent-q5jzr\" (UID: \"24413672-bc61-4434-aa07-40013e6257a6\") " pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:33:51.572588 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.572558    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw4f6\" (UniqueName: \"kubernetes.io/projected/234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44-kube-api-access-rw4f6\") pod \"multus-additional-cni-plugins-xtpwr\" (UID: \"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44\") " pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.573241 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.573219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptcnz\" (UniqueName: \"kubernetes.io/projected/eed4bcfc-3ec4-4eb2-8240-78330f59aae2-kube-api-access-ptcnz\") pod \"aws-ebs-csi-driver-node-9l68p\" (UID: \"eed4bcfc-3ec4-4eb2-8240-78330f59aae2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.573976 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.573955    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk6px\" (UniqueName: \"kubernetes.io/projected/259d37f7-cfb6-46f5-91aa-dc6f9e7fb505-kube-api-access-wk6px\") pod \"ovnkube-node-g4p2b\" (UID: \"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.577115 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.577062    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pr5j\" (UniqueName: \"kubernetes.io/projected/bd3d4868-8036-4b10-b7ce-15d43f5851e1-kube-api-access-8pr5j\") pod \"iptables-alerter-xdmr5\" (UID: \"bd3d4868-8036-4b10-b7ce-15d43f5851e1\") " pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.584701 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.582094    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:33:51.584701 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.582137    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:33:51.584701 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.582148    2578 projected.go:196] Error preparing data for projected volume kube-api-access-zks2p for pod openshift-network-diagnostics/network-check-target-4g95m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:51.584701 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:51.582216    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p podName:77f6e728-ad76-4515-a9d5-b13d3b72967e nodeName:}" failed. No retries permitted until 2026-07-27 21:33:52.082191683 +0000 UTC m=+2.978001120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zks2p" (UniqueName: "kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p") pod "network-check-target-4g95m" (UID: "77f6e728-ad76-4515-a9d5-b13d3b72967e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:51.584701 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.584243    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znl9f\" (UniqueName: \"kubernetes.io/projected/d32a5315-4ac2-4d35-b133-854854da98c7-kube-api-access-znl9f\") pod \"node-ca-22g6x\" (UID: \"d32a5315-4ac2-4d35-b133-854854da98c7\") " pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.585186 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.585159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf62g\" (UniqueName: \"kubernetes.io/projected/4057d2d1-dd7e-46e1-af61-4a26a094ca35-kube-api-access-nf62g\") pod \"multus-qgbwm\" (UID: \"4057d2d1-dd7e-46e1-af61-4a26a094ca35\") " pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.585319 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.585269    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkx8s\" (UniqueName: \"kubernetes.io/projected/4ea190da-d6af-4616-93a3-4ad59b61282a-kube-api-access-vkx8s\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:51.604855 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.604813    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal" event={"ID":"0c7066e91c3e4c6ab197571918f76a0f","Type":"ContainerStarted","Data":"9b1f163008503c7053f4f860acd0b7d5549214dad9ea27638d08faebd99186a9"}
Jul 27 21:33:51.606250 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.606204    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal" event={"ID":"19538ed6d308dc3f353d7b9a9b7dce11","Type":"ContainerStarted","Data":"abd40f322c29186063960ec5d796c1830bc2b5a7d01f5d2f1a56a2568065a3a2"}
Jul 27 21:33:51.647752 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.647710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-sysctl-d\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.647866 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.647768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-lib-modules\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.647866 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.647793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-modprobe-d\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.647985 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.647915    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-modprobe-d\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.647985 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.647920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-sysctl-d\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.647985 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.647941    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-lib-modules\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.647985 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.647960    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b29ce2d-9385-4403-8d0f-554572628d9a-tmp\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f9f3fc-17d0-4b1d-8a50-01757a4f934c-tmp-dir\") pod \"node-resolver-ftzbq\" (UID: \"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c\") " pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.648182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-systemd\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648075    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-host\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-systemd\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-kubernetes\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-tuned\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648144    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-host\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tznl\" (UniqueName: \"kubernetes.io/projected/c9f9f3fc-17d0-4b1d-8a50-01757a4f934c-kube-api-access-8tznl\") pod \"node-resolver-ftzbq\" (UID: \"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c\") " pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-run\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-kubernetes\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-var-lib-kubelet\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-var-lib-kubelet\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72lrs\" (UniqueName: \"kubernetes.io/projected/2b29ce2d-9385-4403-8d0f-554572628d9a-kube-api-access-72lrs\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648300    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f9f3fc-17d0-4b1d-8a50-01757a4f934c-tmp-dir\") pod \"node-resolver-ftzbq\" (UID: \"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c\") " pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648318    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-sys\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-sysconfig\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648360    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-sys\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648422    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-run\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648425    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-sysconfig\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-sysctl-conf\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.648541 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9f9f3fc-17d0-4b1d-8a50-01757a4f934c-hosts-file\") pod \"node-resolver-ftzbq\" (UID: \"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c\") " pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.649004 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648580    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9f9f3fc-17d0-4b1d-8a50-01757a4f934c-hosts-file\") pod \"node-resolver-ftzbq\" (UID: \"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c\") " pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.649004 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.648630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-sysctl-conf\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.651131 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.651107    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b29ce2d-9385-4403-8d0f-554572628d9a-tmp\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.651303 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.651285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b29ce2d-9385-4403-8d0f-554572628d9a-etc-tuned\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.657608 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.657581    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tznl\" (UniqueName: \"kubernetes.io/projected/c9f9f3fc-17d0-4b1d-8a50-01757a4f934c-kube-api-access-8tznl\") pod \"node-resolver-ftzbq\" (UID: \"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c\") " pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.657764 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.657746    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72lrs\" (UniqueName: \"kubernetes.io/projected/2b29ce2d-9385-4403-8d0f-554572628d9a-kube-api-access-72lrs\") pod \"tuned-2lb6z\" (UID: \"2b29ce2d-9385-4403-8d0f-554572628d9a\") " pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.738335 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.738268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:33:51.748380 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.748357    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xtpwr"
Jul 27 21:33:51.758201 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.758172    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-22g6x"
Jul 27 21:33:51.766177 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.766158    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:33:51.776021 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.775645    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qgbwm"
Jul 27 21:33:51.782704 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.782544    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p"
Jul 27 21:33:51.790811 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.790790    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xdmr5"
Jul 27 21:33:51.799747 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.799726    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2lb6z"
Jul 27 21:33:51.816623 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.816600    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ftzbq"
Jul 27 21:33:51.863279 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:51.863254    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:33:52.152165 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:52.152082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:52.152165 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:52.152152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:52.152366 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:52.152192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:52.152366 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:52.152269    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:52.152366 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:52.152297    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:52.152366 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:52.152333    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:33:52.152366 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:52.152352    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:33:52.152366 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:52.152366    2578 projected.go:196] Error preparing data for projected volume kube-api-access-zks2p for pod openshift-network-diagnostics/network-check-target-4g95m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:52.152588 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:52.152351    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret podName:d3b83b76-20e7-43e7-85bd-885ab2b22ce8 nodeName:}" failed. No retries permitted until 2026-07-27 21:33:53.15233049 +0000 UTC m=+4.048139947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret") pod "global-pull-secret-syncer-2q7dv" (UID: "d3b83b76-20e7-43e7-85bd-885ab2b22ce8") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:52.152588 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:52.152413    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs podName:4ea190da-d6af-4616-93a3-4ad59b61282a nodeName:}" failed. No retries permitted until 2026-07-27 21:33:53.152400974 +0000 UTC m=+4.048210405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs") pod "network-metrics-daemon-wtmrw" (UID: "4ea190da-d6af-4616-93a3-4ad59b61282a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:52.152588 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:52.152428    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p podName:77f6e728-ad76-4515-a9d5-b13d3b72967e nodeName:}" failed. No retries permitted until 2026-07-27 21:33:53.152419888 +0000 UTC m=+4.048229318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zks2p" (UniqueName: "kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p") pod "network-check-target-4g95m" (UID: "77f6e728-ad76-4515-a9d5-b13d3b72967e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:52.493596 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:52.493499    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 21:28:52 +0000 UTC" deadline="2027-12-26 04:34:10.510583088 +0000 UTC"
Jul 27 21:33:52.493596 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:52.493549    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12391h0m18.017036768s"
Jul 27 21:33:52.810081 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:52.810031    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f9f3fc_17d0_4b1d_8a50_01757a4f934c.slice/crio-c0786c0cd61770ca3c482af0349891ff7c7d63c19e853e76ab57bab4b7979d80 WatchSource:0}: Error finding container c0786c0cd61770ca3c482af0349891ff7c7d63c19e853e76ab57bab4b7979d80: Status 404 returned error can't find the container with id c0786c0cd61770ca3c482af0349891ff7c7d63c19e853e76ab57bab4b7979d80
Jul 27 21:33:52.817889 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:52.817839    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259d37f7_cfb6_46f5_91aa_dc6f9e7fb505.slice/crio-c7399e17cc66726927919eea61d7c543c23366947b69e4c0f2cc7890cbdb862f WatchSource:0}: Error finding container c7399e17cc66726927919eea61d7c543c23366947b69e4c0f2cc7890cbdb862f: Status 404 returned error can't find the container with id c7399e17cc66726927919eea61d7c543c23366947b69e4c0f2cc7890cbdb862f
Jul 27 21:33:52.819369 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:52.819343    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4057d2d1_dd7e_46e1_af61_4a26a094ca35.slice/crio-7636e374d91819556957e1d3c9b9c9ffd5668aa347bb7b4b20a550a8d0c43203 WatchSource:0}: Error finding container 7636e374d91819556957e1d3c9b9c9ffd5668aa347bb7b4b20a550a8d0c43203: Status 404 returned error can't find the container with id 7636e374d91819556957e1d3c9b9c9ffd5668aa347bb7b4b20a550a8d0c43203
Jul 27 21:33:52.820182 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:52.820131    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32a5315_4ac2_4d35_b133_854854da98c7.slice/crio-a9338e2f4862ab412f07dca5b8d57815ae4200f305d4df1d734e7d6d77a91024 WatchSource:0}: Error finding container a9338e2f4862ab412f07dca5b8d57815ae4200f305d4df1d734e7d6d77a91024: Status 404 returned error can't find the container with id a9338e2f4862ab412f07dca5b8d57815ae4200f305d4df1d734e7d6d77a91024
Jul 27 21:33:52.821689 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:52.821510    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b29ce2d_9385_4403_8d0f_554572628d9a.slice/crio-9418d5319c081e1f4c956608d5f99b63e9744cb7500b17703d6db0691a7b2583 WatchSource:0}: Error finding container 9418d5319c081e1f4c956608d5f99b63e9744cb7500b17703d6db0691a7b2583: Status 404 returned error can't find the container with id 9418d5319c081e1f4c956608d5f99b63e9744cb7500b17703d6db0691a7b2583
Jul 27 21:33:52.823929 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:52.823884    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed4bcfc_3ec4_4eb2_8240_78330f59aae2.slice/crio-d817e5ddbdbe5255a64475148bcc77eea0a0c206c24f59b3e454bb91d8446495 WatchSource:0}: Error finding container d817e5ddbdbe5255a64475148bcc77eea0a0c206c24f59b3e454bb91d8446495: Status 404 returned error can't find the container with id d817e5ddbdbe5255a64475148bcc77eea0a0c206c24f59b3e454bb91d8446495
Jul 27 21:33:52.824964 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:33:52.824939    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd3d4868_8036_4b10_b7ce_15d43f5851e1.slice/crio-42ed0ffd3d35ce02de1da3c2d01779d8622a292d2be0050fc3f8aac0920f27e8 WatchSource:0}: Error finding container 42ed0ffd3d35ce02de1da3c2d01779d8622a292d2be0050fc3f8aac0920f27e8: Status 404 returned error can't find the container with id 42ed0ffd3d35ce02de1da3c2d01779d8622a292d2be0050fc3f8aac0920f27e8
Jul 27 21:33:53.158702 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.158668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:53.158835 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.158726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:53.158884 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.158854    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:33:53.158884 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.158860    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:53.158884 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.158870    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:33:53.159003 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.158889    2578 projected.go:196] Error preparing data for projected volume kube-api-access-zks2p for pod openshift-network-diagnostics/network-check-target-4g95m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:53.159003 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.158927    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret podName:d3b83b76-20e7-43e7-85bd-885ab2b22ce8 nodeName:}" failed. No retries permitted until 2026-07-27 21:33:55.158906247 +0000 UTC m=+6.054715699 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret") pod "global-pull-secret-syncer-2q7dv" (UID: "d3b83b76-20e7-43e7-85bd-885ab2b22ce8") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:53.159003 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.158955    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:53.159159 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.159007    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p podName:77f6e728-ad76-4515-a9d5-b13d3b72967e nodeName:}" failed. No retries permitted until 2026-07-27 21:33:55.1589939 +0000 UTC m=+6.054803336 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zks2p" (UniqueName: "kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p") pod "network-check-target-4g95m" (UID: "77f6e728-ad76-4515-a9d5-b13d3b72967e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:53.159159 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.159078    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:53.159159 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.159141    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs podName:4ea190da-d6af-4616-93a3-4ad59b61282a nodeName:}" failed. No retries permitted until 2026-07-27 21:33:55.159122952 +0000 UTC m=+6.054932400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs") pod "network-metrics-daemon-wtmrw" (UID: "4ea190da-d6af-4616-93a3-4ad59b61282a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:53.493888 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.493744    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 21:28:52 +0000 UTC" deadline="2028-03-12 18:54:13.073613579 +0000 UTC"
Jul 27 21:33:53.493888 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.493779    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14253h20m19.579840096s"
Jul 27 21:33:53.603572 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.602481    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:53.603572 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.602607    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:33:53.603572 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.603041    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:53.603572 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.603154    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:33:53.603572 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.603221    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:53.603572 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:53.603293    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:33:53.632849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.632816    2578 generic.go:356] "Generic (PLEG): container finished" podID="0c7066e91c3e4c6ab197571918f76a0f" containerID="f80eb007feb32e2597cae3e6fe30c4da344dd06ff930878834ca98e2493a8419" exitCode=0
Jul 27 21:33:53.632981 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.632906    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal" event={"ID":"0c7066e91c3e4c6ab197571918f76a0f","Type":"ContainerDied","Data":"f80eb007feb32e2597cae3e6fe30c4da344dd06ff930878834ca98e2493a8419"}
Jul 27 21:33:53.636136 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.636109    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xdmr5" event={"ID":"bd3d4868-8036-4b10-b7ce-15d43f5851e1","Type":"ContainerStarted","Data":"42ed0ffd3d35ce02de1da3c2d01779d8622a292d2be0050fc3f8aac0920f27e8"}
Jul 27 21:33:53.643556 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.643527    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-22g6x" event={"ID":"d32a5315-4ac2-4d35-b133-854854da98c7","Type":"ContainerStarted","Data":"a9338e2f4862ab412f07dca5b8d57815ae4200f305d4df1d734e7d6d77a91024"}
Jul 27 21:33:53.650348 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.650319    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qgbwm" event={"ID":"4057d2d1-dd7e-46e1-af61-4a26a094ca35","Type":"ContainerStarted","Data":"7636e374d91819556957e1d3c9b9c9ffd5668aa347bb7b4b20a550a8d0c43203"}
Jul 27 21:33:53.658936 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.658847    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ftzbq" event={"ID":"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c","Type":"ContainerStarted","Data":"c0786c0cd61770ca3c482af0349891ff7c7d63c19e853e76ab57bab4b7979d80"}
Jul 27 21:33:53.660733 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.660673    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q5jzr" event={"ID":"24413672-bc61-4434-aa07-40013e6257a6","Type":"ContainerStarted","Data":"f6707d67ac38b88bdf47362931acb3941912ad5ff597e6fa1780b02264c3de68"}
Jul 27 21:33:53.664710 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.664165    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal" event={"ID":"19538ed6d308dc3f353d7b9a9b7dce11","Type":"ContainerStarted","Data":"cc5d378bf3ea76a8ceac3187ab57d0b0a180b28b854d369a33ffb38032bbb268"}
Jul 27 21:33:53.678578 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.675649    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p" event={"ID":"eed4bcfc-3ec4-4eb2-8240-78330f59aae2","Type":"ContainerStarted","Data":"d817e5ddbdbe5255a64475148bcc77eea0a0c206c24f59b3e454bb91d8446495"}
Jul 27 21:33:53.678578 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.677558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2lb6z" event={"ID":"2b29ce2d-9385-4403-8d0f-554572628d9a","Type":"ContainerStarted","Data":"9418d5319c081e1f4c956608d5f99b63e9744cb7500b17703d6db0691a7b2583"}
Jul 27 21:33:53.681449 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.680739    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" event={"ID":"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505","Type":"ContainerStarted","Data":"c7399e17cc66726927919eea61d7c543c23366947b69e4c0f2cc7890cbdb862f"}
Jul 27 21:33:53.685676 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.685612    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtpwr" event={"ID":"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44","Type":"ContainerStarted","Data":"3176efc6530b9942682f17f3fc0c5f6ea7e06a195a379bdaf9149b528a0b2429"}
Jul 27 21:33:53.687105 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:53.686440    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-161.ec2.internal" podStartSLOduration=2.6864236 podStartE2EDuration="2.6864236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:33:53.686105026 +0000 UTC m=+4.581914481" watchObservedRunningTime="2026-07-27 21:33:53.6864236 +0000 UTC m=+4.582233055"
Jul 27 21:33:54.706226 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:54.706149    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal" event={"ID":"0c7066e91c3e4c6ab197571918f76a0f","Type":"ContainerStarted","Data":"79f4368f8f141863da1fe4458b81493dd54f5b02bfcee9aac23cb9a4b48b860f"}
Jul 27 21:33:55.177332 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:55.177225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:55.177332 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:55.177291    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:55.177332 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:55.177322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:55.177627 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.177448    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:55.177627 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.177520    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs podName:4ea190da-d6af-4616-93a3-4ad59b61282a nodeName:}" failed. No retries permitted until 2026-07-27 21:33:59.177501515 +0000 UTC m=+10.073310952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs") pod "network-metrics-daemon-wtmrw" (UID: "4ea190da-d6af-4616-93a3-4ad59b61282a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:55.177991 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.177968    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:55.178095 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.178031    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret podName:d3b83b76-20e7-43e7-85bd-885ab2b22ce8 nodeName:}" failed. No retries permitted until 2026-07-27 21:33:59.178014826 +0000 UTC m=+10.073824271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret") pod "global-pull-secret-syncer-2q7dv" (UID: "d3b83b76-20e7-43e7-85bd-885ab2b22ce8") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:55.178160 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.178115    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:33:55.178160 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.178134    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:33:55.178160 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.178147    2578 projected.go:196] Error preparing data for projected volume kube-api-access-zks2p for pod openshift-network-diagnostics/network-check-target-4g95m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:55.178314 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.178188    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p podName:77f6e728-ad76-4515-a9d5-b13d3b72967e nodeName:}" failed. No retries permitted until 2026-07-27 21:33:59.178174524 +0000 UTC m=+10.073983960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zks2p" (UniqueName: "kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p") pod "network-check-target-4g95m" (UID: "77f6e728-ad76-4515-a9d5-b13d3b72967e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:55.602032 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:55.601939    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:55.602188 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.602089    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:33:55.602188 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:55.602145    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:55.602305 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.602231    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:33:55.602305 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:55.602285    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:55.602428 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:55.602347    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:33:57.599953 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:57.599467    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:57.599953 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:57.599506    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:57.599953 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:57.599467    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:57.599953 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:57.599622    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:33:57.599953 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:57.599783    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:33:57.599953 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:57.599893    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:59.209316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:59.209373    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:59.209429    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.209551    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.209613    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret podName:d3b83b76-20e7-43e7-85bd-885ab2b22ce8 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:07.20959423 +0000 UTC m=+18.105403666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret") pod "global-pull-secret-syncer-2q7dv" (UID: "d3b83b76-20e7-43e7-85bd-885ab2b22ce8") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.210060    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.210079    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.210092    2578 projected.go:196] Error preparing data for projected volume kube-api-access-zks2p for pod openshift-network-diagnostics/network-check-target-4g95m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.210138    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p podName:77f6e728-ad76-4515-a9d5-b13d3b72967e nodeName:}" failed. No retries permitted until 2026-07-27 21:34:07.210123949 +0000 UTC m=+18.105933398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zks2p" (UniqueName: "kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p") pod "network-check-target-4g95m" (UID: "77f6e728-ad76-4515-a9d5-b13d3b72967e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.210205    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:59.210272 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.210241    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs podName:4ea190da-d6af-4616-93a3-4ad59b61282a nodeName:}" failed. No retries permitted until 2026-07-27 21:34:07.210228462 +0000 UTC m=+18.106037893 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs") pod "network-metrics-daemon-wtmrw" (UID: "4ea190da-d6af-4616-93a3-4ad59b61282a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:33:59.600469 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:59.600384    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:33:59.600631 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.600504    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:33:59.600631 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:59.600521    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:33:59.600631 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:33:59.600527    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:33:59.600631 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.600602    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:33:59.600831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:33:59.600680    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:01.600302 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:01.600092    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:01.600761 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:01.600418    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:01.600761 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:01.600265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:01.600761 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:01.600511    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:01.600761 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:01.600265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:01.600761 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:01.600587    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:03.599625 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:03.599590    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:03.600065 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:03.599590    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:03.600065 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:03.599708    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:03.600065 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:03.599590    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:03.600065 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:03.599818    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:03.600065 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:03.599899    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:05.599747 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:05.599698    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:05.599747 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:05.599698    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:05.600232 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:05.599707    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:05.600232 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:05.599845    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:05.600232 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:05.599936    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:05.600232 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:05.599997    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:07.272818 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:07.272779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:07.272857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:07.272893    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.272930    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.272998    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.273018    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.273002    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret podName:d3b83b76-20e7-43e7-85bd-885ab2b22ce8 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:23.27298158 +0000 UTC m=+34.168791026 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret") pod "global-pull-secret-syncer-2q7dv" (UID: "d3b83b76-20e7-43e7-85bd-885ab2b22ce8") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.273036    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.273046    2578 projected.go:196] Error preparing data for projected volume kube-api-access-zks2p for pod openshift-network-diagnostics/network-check-target-4g95m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.273048    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs podName:4ea190da-d6af-4616-93a3-4ad59b61282a nodeName:}" failed. No retries permitted until 2026-07-27 21:34:23.273035977 +0000 UTC m=+34.168845406 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs") pod "network-metrics-daemon-wtmrw" (UID: "4ea190da-d6af-4616-93a3-4ad59b61282a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:34:07.273369 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.273084    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p podName:77f6e728-ad76-4515-a9d5-b13d3b72967e nodeName:}" failed. No retries permitted until 2026-07-27 21:34:23.273072069 +0000 UTC m=+34.168881501 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zks2p" (UniqueName: "kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p") pod "network-check-target-4g95m" (UID: "77f6e728-ad76-4515-a9d5-b13d3b72967e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:34:07.600219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:07.600141    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:07.600219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:07.600156    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:07.600219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:07.600156    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:07.600507 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.600255    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:07.600507 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.600326    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:07.600507 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:07.600402    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:09.600703 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:09.600674    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:09.601198 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:09.600797    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:09.601198 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:09.600878    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:09.601198 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:09.600979    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:09.601198 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:09.601016    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:09.601198 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:09.601082    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:11.599627 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:11.599595    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:11.600161 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:11.599595    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:11.600161 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:11.599711    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:11.600161 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:11.599612    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:11.600161 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:11.599836    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:11.600161 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:11.599903    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:12.745604 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.745274    2578 generic.go:356] "Generic (PLEG): container finished" podID="234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44" containerID="11ca1b40af2af9cc58cd85715f06624d0313cf9028e018a373961403769ff2b0" exitCode=0
Jul 27 21:34:12.746498 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.745362    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtpwr" event={"ID":"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44","Type":"ContainerDied","Data":"11ca1b40af2af9cc58cd85715f06624d0313cf9028e018a373961403769ff2b0"}
Jul 27 21:34:12.747353 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.747324    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-22g6x" event={"ID":"d32a5315-4ac2-4d35-b133-854854da98c7","Type":"ContainerStarted","Data":"72bd0f09bedc96ab58f3e5df75dc8fa838e26a730e9353c3fb6a11e1bd394b2d"}
Jul 27 21:34:12.748622 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.748599    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qgbwm" event={"ID":"4057d2d1-dd7e-46e1-af61-4a26a094ca35","Type":"ContainerStarted","Data":"c439a99b88083a87f6154b9662820afabdd2be858b5fab301cd7d884a812d3b6"}
Jul 27 21:34:12.749862 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.749828    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ftzbq" event={"ID":"c9f9f3fc-17d0-4b1d-8a50-01757a4f934c","Type":"ContainerStarted","Data":"da2a627e965f05a955c343d2bf67ebb38621676fbd6499897b2c88c72bf316cf"}
Jul 27 21:34:12.751311 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.751287    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q5jzr" event={"ID":"24413672-bc61-4434-aa07-40013e6257a6","Type":"ContainerStarted","Data":"579ac3631ceae695b93fc3a563dfe13458e1bb9f642e968ee874905c8bea553b"}
Jul 27 21:34:12.752853 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.752834    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p" event={"ID":"eed4bcfc-3ec4-4eb2-8240-78330f59aae2","Type":"ContainerStarted","Data":"a0a922302451d791a20f10cd7553f0ec35e2a3c59c372f7e935f85f4cf732cdc"}
Jul 27 21:34:12.754217 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.754194    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2lb6z" event={"ID":"2b29ce2d-9385-4403-8d0f-554572628d9a","Type":"ContainerStarted","Data":"1d0f34d4995ed991366b78c4397614721a83e39c12882de205225396a73e1f79"}
Jul 27 21:34:12.757024 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.757002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" event={"ID":"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505","Type":"ContainerStarted","Data":"1d005e577beff1d872170c8b24c9c4135174b495c44be7123fc3f158ac55e0fd"}
Jul 27 21:34:12.757109 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.757074    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" event={"ID":"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505","Type":"ContainerStarted","Data":"74aae493bc65cdd81511d520d04cd610f22528bbf3f8f526068e769cad411c42"}
Jul 27 21:34:12.757160 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.757117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" event={"ID":"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505","Type":"ContainerStarted","Data":"992d206e3e47e055e4d0dc141e0133bd33b6031df0cf88405b8bcb4a9d8590ce"}
Jul 27 21:34:12.757160 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.757147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" event={"ID":"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505","Type":"ContainerStarted","Data":"7ebdca8600afefbf594352f51a6adfc68efc1e9cc839246d1f7dfb47e251c8eb"}
Jul 27 21:34:12.757237 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.757181    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" event={"ID":"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505","Type":"ContainerStarted","Data":"c94a90ff8129edb8fb7678c7c36f473d6042f55d3b7acf022739b3fba1fcb172"}
Jul 27 21:34:12.757237 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.757226    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" event={"ID":"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505","Type":"ContainerStarted","Data":"1770db73cffd23c8ada34778a5a6b15dd2babd36fbdc2bc0cbf949b70693301c"}
Jul 27 21:34:12.771836 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.771789    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-161.ec2.internal" podStartSLOduration=21.7717781 podStartE2EDuration="21.7717781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:33:54.723559303 +0000 UTC m=+5.619368759" watchObservedRunningTime="2026-07-27 21:34:12.7717781 +0000 UTC m=+23.667587549"
Jul 27 21:34:12.786282 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.786250    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q5jzr" podStartSLOduration=7.858036104 podStartE2EDuration="22.786236933s" totalImagesPullingTime="14.928200829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:12.786238281 +0000 UTC m=+23.682047743" watchObservedRunningTime="2026-07-27 21:34:12.786236933 +0000 UTC m=+23.682046378"
Jul 27 21:34:12.802337 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.802306    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2lb6z" podStartSLOduration=3.836941073 podStartE2EDuration="22.802296187s" totalImagesPullingTime="18.965355114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:12.802189064 +0000 UTC m=+23.697998515" watchObservedRunningTime="2026-07-27 21:34:12.802296187 +0000 UTC m=+23.698105637"
Jul 27 21:34:12.818234 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.818204    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-22g6x" podStartSLOduration=3.874871862 podStartE2EDuration="22.818193793s" totalImagesPullingTime="18.943321931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:12.8176697 +0000 UTC m=+23.713479151" watchObservedRunningTime="2026-07-27 21:34:12.818193793 +0000 UTC m=+23.714003243"
Jul 27 21:34:12.840514 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.840484    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qgbwm" podStartSLOduration=3.841871969 podStartE2EDuration="22.840476622s" totalImagesPullingTime="18.998604653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:12.839664317 +0000 UTC m=+23.735473811" watchObservedRunningTime="2026-07-27 21:34:12.840476622 +0000 UTC m=+23.736286072"
Jul 27 21:34:12.862600 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:12.862569    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ftzbq" podStartSLOduration=3.912568937 podStartE2EDuration="22.862562668s" totalImagesPullingTime="18.949993731s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:12.862385064 +0000 UTC m=+23.758194516" watchObservedRunningTime="2026-07-27 21:34:12.862562668 +0000 UTC m=+23.758372119"
Jul 27 21:34:13.308498 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.308478    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 21:34:13.504921 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.504819    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T21:34:13.308494254Z","UUID":"97a099c1-8e1a-4a77-b353-617d4ba28496","Handler":null,"Name":"","Endpoint":""}
Jul 27 21:34:13.506179 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.506162    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 21:34:13.506275 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.506201    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 21:34:13.599475 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.599449    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:13.599610 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.599459    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:13.599610 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:13.599550    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:13.599610 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.599576    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:13.599747 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:13.599649    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:13.599747 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:13.599704    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:13.761243 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.761169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xdmr5" event={"ID":"bd3d4868-8036-4b10-b7ce-15d43f5851e1","Type":"ContainerStarted","Data":"90969bcc586496b80a5512800ac603102049955a0696f64d873eab0dc39badb8"}
Jul 27 21:34:13.763291 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.763266    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p" event={"ID":"eed4bcfc-3ec4-4eb2-8240-78330f59aae2","Type":"ContainerStarted","Data":"1cef2c595630ff98e5348c48851a098ff0fb8caa96acee5a26f5adac9d27d3f1"}
Jul 27 21:34:13.776947 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:13.776910    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xdmr5" podStartSLOduration=4.833452522 podStartE2EDuration="23.776894991s" totalImagesPullingTime="18.943442469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:13.775724546 +0000 UTC m=+24.671533998" watchObservedRunningTime="2026-07-27 21:34:13.776894991 +0000 UTC m=+24.672704443"
Jul 27 21:34:14.768371 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:14.768086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p" event={"ID":"eed4bcfc-3ec4-4eb2-8240-78330f59aae2","Type":"ContainerStarted","Data":"06caab34042cfb07431629e59c04ec81120befc4b7ba88c34448e2ec31108f10"}
Jul 27 21:34:14.771694 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:14.771665    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" event={"ID":"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505","Type":"ContainerStarted","Data":"64e7be0ef5d288f9b0e7f198b8ba60ff5d5f6cf8610d7790f77aeaaab9c00bf2"}
Jul 27 21:34:14.792692 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:14.792652    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9l68p" podStartSLOduration=3.42763286 podStartE2EDuration="24.79262007s" totalImagesPullingTime="21.36498721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:14.791518706 +0000 UTC m=+25.687328158" watchObservedRunningTime="2026-07-27 21:34:14.79262007 +0000 UTC m=+25.688429522"
Jul 27 21:34:15.599379 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:15.599351    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:15.599647 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:15.599349    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:15.599647 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:15.599457    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:15.599647 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:15.599543    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:15.599647 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:15.599580    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:15.599894 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:15.599649    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:15.678670 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:15.678643    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:34:15.679364 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:15.679222    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:34:17.599595 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.599562    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:17.600323 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.599566    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:17.600323 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:17.599668    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:17.600323 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.599567    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:17.600323 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:17.599768    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:17.600323 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:17.599871    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:17.781222 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.781162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" event={"ID":"259d37f7-cfb6-46f5-91aa-dc6f9e7fb505","Type":"ContainerStarted","Data":"d85716d08c12ccb5d570265e7a2bfab8f0956937b157fe931e71a9bc799a11d6"}
Jul 27 21:34:17.781424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.781396    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:34:17.781544 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.781465    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:34:17.783007 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.782986    2578 generic.go:356] "Generic (PLEG): container finished" podID="234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44" containerID="44023bf5402b386c77160797c9f5267353544fb654a249a6cec5f718f80b1ab1" exitCode=0
Jul 27 21:34:17.783110 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.783014    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtpwr" event={"ID":"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44","Type":"ContainerDied","Data":"44023bf5402b386c77160797c9f5267353544fb654a249a6cec5f718f80b1ab1"}
Jul 27 21:34:17.797322 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.797305    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:34:17.849769 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:17.849710    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b" podStartSLOduration=8.827347617000001 podStartE2EDuration="27.849698688s" totalImagesPullingTime="19.022351071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:17.816877009 +0000 UTC m=+28.712686605" watchObservedRunningTime="2026-07-27 21:34:17.849698688 +0000 UTC m=+28.745508139"
Jul 27 21:34:18.785393 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:18.785206    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:34:18.800171 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:18.800137    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:34:19.235442 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:19.235415    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2q7dv"]
Jul 27 21:34:19.235586 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:19.235520    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:19.235633 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:19.235615    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:19.238468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:19.238443    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4g95m"]
Jul 27 21:34:19.238606 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:19.238534    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:19.238676 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:19.238611    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:19.250651 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:19.250632    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wtmrw"]
Jul 27 21:34:19.250728 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:19.250714    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:19.250840 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:19.250823    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:19.788577 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:19.788543    2578 generic.go:356] "Generic (PLEG): container finished" podID="234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44" containerID="37817beb3fa4f4a5f770e329f89dd6a8190f0d196d7b9b0007882949c7fa86ca" exitCode=0
Jul 27 21:34:19.789012 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:19.788625    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtpwr" event={"ID":"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44","Type":"ContainerDied","Data":"37817beb3fa4f4a5f770e329f89dd6a8190f0d196d7b9b0007882949c7fa86ca"}
Jul 27 21:34:20.317828 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:20.317801    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:34:20.317963 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:20.317952    2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 27 21:34:20.318474 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:20.318452    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q5jzr"
Jul 27 21:34:20.600104 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:20.600050    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:20.600104 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:20.600073    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:20.600104 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:20.600079    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:20.600273 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:20.600153    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:20.600273 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:20.600236    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:20.600339 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:20.600297    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:21.795491 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:21.795301    2578 generic.go:356] "Generic (PLEG): container finished" podID="234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44" containerID="cd156a2748f30458aa7e8537aa0f535035d82ddf7b8e0e7dc4692792a74e3c31" exitCode=0
Jul 27 21:34:21.795848 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:21.795386    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtpwr" event={"ID":"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44","Type":"ContainerDied","Data":"cd156a2748f30458aa7e8537aa0f535035d82ddf7b8e0e7dc4692792a74e3c31"}
Jul 27 21:34:22.599920 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:22.599890    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:22.600103 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:22.599989    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:22.600103 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:22.600001    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4g95m" podUID="77f6e728-ad76-4515-a9d5-b13d3b72967e"
Jul 27 21:34:22.600103 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:22.600007    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:22.600103 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:22.600091    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q7dv" podUID="d3b83b76-20e7-43e7-85bd-885ab2b22ce8"
Jul 27 21:34:22.600321 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:22.600184    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wtmrw" podUID="4ea190da-d6af-4616-93a3-4ad59b61282a"
Jul 27 21:34:23.296389 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:23.296355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:23.296403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:23.296431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:23.296519    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:23.296527    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:23.296542    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:23.296563    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:23.296571    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs podName:4ea190da-d6af-4616-93a3-4ad59b61282a nodeName:}" failed. No retries permitted until 2026-07-27 21:34:55.296557925 +0000 UTC m=+66.192367353 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs") pod "network-metrics-daemon-wtmrw" (UID: "4ea190da-d6af-4616-93a3-4ad59b61282a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:23.296575    2578 projected.go:196] Error preparing data for projected volume kube-api-access-zks2p for pod openshift-network-diagnostics/network-check-target-4g95m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:23.296588    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret podName:d3b83b76-20e7-43e7-85bd-885ab2b22ce8 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:55.296578539 +0000 UTC m=+66.192387968 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret") pod "global-pull-secret-syncer-2q7dv" (UID: "d3b83b76-20e7-43e7-85bd-885ab2b22ce8") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:34:23.296854 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:23.296625    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p podName:77f6e728-ad76-4515-a9d5-b13d3b72967e nodeName:}" failed. No retries permitted until 2026-07-27 21:34:55.29660699 +0000 UTC m=+66.192416419 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zks2p" (UniqueName: "kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p") pod "network-check-target-4g95m" (UID: "77f6e728-ad76-4515-a9d5-b13d3b72967e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:34:24.422770 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.421569    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeReady"
Jul 27 21:34:24.422770 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.421715    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 21:34:24.460096 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.460043    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59cd4b4456-zprqx"]
Jul 27 21:34:24.462986 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.462453    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"]
Jul 27 21:34:24.462986 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.462968    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.463970 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.463932    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64884fdbff-j7wtj"]
Jul 27 21:34:24.464083 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.464044    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:24.466898 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.466878    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh"]
Jul 27 21:34:24.467054 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.467033    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.467147 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.467081    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zzpc9\""
Jul 27 21:34:24.467232 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.467209    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 27 21:34:24.467404 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.467386    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 27 21:34:24.468061 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.468041    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 27 21:34:24.469010 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.468668    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"]
Jul 27 21:34:24.469010 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.468682    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.469010 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.468727    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 27 21:34:24.469010 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.468753    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.469010 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.468760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh"
Jul 27 21:34:24.469010 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.468685    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6vht4\""
Jul 27 21:34:24.469010 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.468863    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 27 21:34:24.470443 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.470416    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 27 21:34:24.470561 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.470476    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.470561 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.470417    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"]
Jul 27 21:34:24.472122 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.472100    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 27 21:34:24.472761 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.472233    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.472761 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.472406    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 27 21:34:24.472761 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.472643    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.472939 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.472767    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.472939 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.472854    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-s9j86\""
Jul 27 21:34:24.473042 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.473020    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.473365 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.473344    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 27 21:34:24.473544 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.473529    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.473898 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.473878    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 27 21:34:24.474002 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.473913    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.474002 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.473922    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 27 21:34:24.474002 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.473979    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z2vzc\""
Jul 27 21:34:24.474289 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.474268    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.474430 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.474410    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9k8kt\""
Jul 27 21:34:24.476416 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.476397    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.476492 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.476462    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-42c4q\""
Jul 27 21:34:24.476530 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.476504    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 27 21:34:24.476862 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.476703    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.476862 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.476703    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 27 21:34:24.477366 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.477346    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 27 21:34:24.477877 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.477856    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"]
Jul 27 21:34:24.480700 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.480465    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qdxcv"]
Jul 27 21:34:24.482938 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.482917    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.489534 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.488355    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"]
Jul 27 21:34:24.490564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.490535    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"]
Jul 27 21:34:24.490662 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.490587    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-cpgfx"]
Jul 27 21:34:24.490716 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.490705    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:24.491675 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.491658    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 21:34:24.492047 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.492026    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kb678\""
Jul 27 21:34:24.497293 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.497270    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 27 21:34:24.499793 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.497748    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-wzbsj"]
Jul 27 21:34:24.499793 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.499114    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.500249 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.500226    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 27 21:34:24.500485 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.500411    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 21:34:24.500655 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.500634    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pfmkx\""
Jul 27 21:34:24.501429 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.501406    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"]
Jul 27 21:34:24.503200 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.503171    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 21:34:24.503374 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.503357    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 21:34:24.503569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.503545    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f5bdw\""
Jul 27 21:34:24.503778 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.503760    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.506031 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.504009    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.506031 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.505640    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.506498 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.506471    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8xsl4"]
Jul 27 21:34:24.506612 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.506592    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:24.508593 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.508573    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 21:34:24.509076 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.509056    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.509394 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.509368    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.509592 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.509572    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 21:34:24.509838 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.509823    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-brxjg\""
Jul 27 21:34:24.511014 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.510542    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 27 21:34:24.511014 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.510671    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.512508 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.512488    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6w5sb\""
Jul 27 21:34:24.512976 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.512955    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.513329 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.513302    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 21:34:24.513698 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.513681    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh"]
Jul 27 21:34:24.514027 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.513901    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64884fdbff-j7wtj"]
Jul 27 21:34:24.514027 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.513941    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59cd4b4456-zprqx"]
Jul 27 21:34:24.514027 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.513792    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8xsl4"
Jul 27 21:34:24.514027 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.514004    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"]
Jul 27 21:34:24.514027 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.514026    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt"]
Jul 27 21:34:24.515942 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.515889    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"]
Jul 27 21:34:24.515942 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.515917    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qdxcv"]
Jul 27 21:34:24.516639 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.515943    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-cpgfx"]
Jul 27 21:34:24.516639 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.515963    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-wzbsj"]
Jul 27 21:34:24.516639 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.515982    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8xsl4"]
Jul 27 21:34:24.516639 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.516000    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt"]
Jul 27 21:34:24.516639 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.516081    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt"
Jul 27 21:34:24.516639 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.516081    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"]
Jul 27 21:34:24.516639 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.516573    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.517142 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.517122    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7fmv4\""
Jul 27 21:34:24.517445 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.517158    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.517590 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.517171    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 21:34:24.518460 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.518443    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 21:34:24.518719 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.518698    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2mvn4\""
Jul 27 21:34:24.518849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.518637    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 21:34:24.518936 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.518650    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 21:34:24.599462 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.599442    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:24.599560 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.599445    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:24.599907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.599443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:24.602424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.602405    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xnkh\""
Jul 27 21:34:24.602508 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.602410    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 21:34:24.602640 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.602625    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcmmw\""
Jul 27 21:34:24.602793 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.602777    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 21:34:24.606129 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22ef61cc-b722-40e9-ad7e-4819d634238f-serving-cert\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.606230 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a511784-485f-4334-b266-e9e7dea58199-tmp\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.606230 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606162    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d42297f-2c9c-4373-afc5-667b897f9c57-cert\") pod \"ingress-canary-8xsl4\" (UID: \"6d42297f-2c9c-4373-afc5-667b897f9c57\") " pod="openshift-ingress-canary/ingress-canary-8xsl4"
Jul 27 21:34:24.606230 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rk2l\" (UniqueName: \"kubernetes.io/projected/6d42297f-2c9c-4373-afc5-667b897f9c57-kube-api-access-2rk2l\") pod \"ingress-canary-8xsl4\" (UID: \"6d42297f-2c9c-4373-afc5-667b897f9c57\") " pod="openshift-ingress-canary/ingress-canary-8xsl4"
Jul 27 21:34:24.606230 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p59fm\" (UniqueName: \"kubernetes.io/projected/34756d4e-1ce5-4140-8585-fae29e941ab3-kube-api-access-p59fm\") pod \"volume-data-source-validator-549c6d54cf-vrdbh\" (UID: \"34756d4e-1ce5-4140-8585-fae29e941ab3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh"
Jul 27 21:34:24.606230 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzfc4\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-kube-api-access-kzfc4\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.606474 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606250    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.606474 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:24.606474 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606294    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22ef61cc-b722-40e9-ad7e-4819d634238f-trusted-ca\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.606474 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606355    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:24.606474 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b4cz\" (UniqueName: \"kubernetes.io/projected/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-kube-api-access-7b4cz\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:24.606474 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606423    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-config-volume\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.606474 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-tmp-dir\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606490    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-installation-pull-secrets\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606514    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0442a5bc-b902-4467-bfc2-508cc0aeab47-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhjw2\" (UniqueName: \"kubernetes.io/projected/22ef61cc-b722-40e9-ad7e-4819d634238f-kube-api-access-nhjw2\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606635    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-bound-sa-token\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606656    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a511784-485f-4334-b266-e9e7dea58199-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3a511784-485f-4334-b266-e9e7dea58199-serving-cert\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606724    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29-config\") pod \"service-ca-operator-5d6c75f84f-zmgn6\" (UID: \"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp4d2\" (UniqueName: \"kubernetes.io/projected/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-kube-api-access-xp4d2\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.606816 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vp56\" (UniqueName: \"kubernetes.io/projected/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-kube-api-access-9vp56\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606826    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3a511784-485f-4334-b266-e9e7dea58199-snapshots\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-image-registry-private-configuration\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7620cb9-8c18-4e77-a04d-df44b78842de-ca-trust-extracted\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chbhk\" (UniqueName: \"kubernetes.io/projected/8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29-kube-api-access-chbhk\") pod \"service-ca-operator-5d6c75f84f-zmgn6\" (UID: \"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-trusted-ca\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29-serving-cert\") pod \"service-ca-operator-5d6c75f84f-zmgn6\" (UID: \"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.606980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2e1b4aee-0b22-46b0-80f7-362fd14d99fc-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c\" (UID: \"2e1b4aee-0b22-46b0-80f7-362fd14d99fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607011    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607038    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607091    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607126    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqphd\" (UniqueName: \"kubernetes.io/projected/fd434c96-8b02-486e-8486-4274ccb1b543-kube-api-access-rqphd\") pod \"network-check-source-7d88cb55d8-qxnvt\" (UID: \"fd434c96-8b02-486e-8486-4274ccb1b543\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607172    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-certificates\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607198    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pxqp\" (UniqueName: \"kubernetes.io/projected/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-kube-api-access-9pxqp\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607221    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2e1b4aee-0b22-46b0-80f7-362fd14d99fc-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c\" (UID: \"2e1b4aee-0b22-46b0-80f7-362fd14d99fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.607270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607258    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22ef61cc-b722-40e9-ad7e-4819d634238f-config\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.608134 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4489r\" (UniqueName: \"kubernetes.io/projected/2e1b4aee-0b22-46b0-80f7-362fd14d99fc-kube-api-access-4489r\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c\" (UID: \"2e1b4aee-0b22-46b0-80f7-362fd14d99fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.608134 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-default-certificate\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.608134 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607395    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzhb9\" (UniqueName: \"kubernetes.io/projected/3a511784-485f-4334-b266-e9e7dea58199-kube-api-access-pzhb9\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.608134 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607430    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.608134 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607472    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-stats-auth\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.608134 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.607503    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a511784-485f-4334-b266-e9e7dea58199-service-ca-bundle\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.707906 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.707864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a511784-485f-4334-b266-e9e7dea58199-tmp\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.707906 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.707905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d42297f-2c9c-4373-afc5-667b897f9c57-cert\") pod \"ingress-canary-8xsl4\" (UID: \"6d42297f-2c9c-4373-afc5-667b897f9c57\") " pod="openshift-ingress-canary/ingress-canary-8xsl4"
Jul 27 21:34:24.708150 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.707928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rk2l\" (UniqueName: \"kubernetes.io/projected/6d42297f-2c9c-4373-afc5-667b897f9c57-kube-api-access-2rk2l\") pod \"ingress-canary-8xsl4\" (UID: \"6d42297f-2c9c-4373-afc5-667b897f9c57\") " pod="openshift-ingress-canary/ingress-canary-8xsl4"
Jul 27 21:34:24.708150 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.707952    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p59fm\" (UniqueName: \"kubernetes.io/projected/34756d4e-1ce5-4140-8585-fae29e941ab3-kube-api-access-p59fm\") pod \"volume-data-source-validator-549c6d54cf-vrdbh\" (UID: \"34756d4e-1ce5-4140-8585-fae29e941ab3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh"
Jul 27 21:34:24.708150 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.707979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzfc4\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-kube-api-access-kzfc4\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.708150 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708002    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.708150 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708033    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:24.708150 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22ef61cc-b722-40e9-ad7e-4819d634238f-trusted-ca\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.708150 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708086    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:24.708150 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b4cz\" (UniqueName: \"kubernetes.io/projected/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-kube-api-access-7b4cz\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:24.708150 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-config-volume\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708184    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-tmp-dir\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-installation-pull-secrets\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0442a5bc-b902-4467-bfc2-508cc0aeab47-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhjw2\" (UniqueName: \"kubernetes.io/projected/22ef61cc-b722-40e9-ad7e-4819d634238f-kube-api-access-nhjw2\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-bound-sa-token\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a511784-485f-4334-b266-e9e7dea58199-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708395    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3a511784-485f-4334-b266-e9e7dea58199-serving-cert\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708417    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29-config\") pod \"service-ca-operator-5d6c75f84f-zmgn6\" (UID: \"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708441    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp4d2\" (UniqueName: \"kubernetes.io/projected/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-kube-api-access-xp4d2\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708469    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vp56\" (UniqueName: \"kubernetes.io/projected/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-kube-api-access-9vp56\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3a511784-485f-4334-b266-e9e7dea58199-snapshots\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-image-registry-private-configuration\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.708569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708549    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7620cb9-8c18-4e77-a04d-df44b78842de-ca-trust-extracted\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708578    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chbhk\" (UniqueName: \"kubernetes.io/projected/8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29-kube-api-access-chbhk\") pod \"service-ca-operator-5d6c75f84f-zmgn6\" (UID: \"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-trusted-ca\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29-serving-cert\") pod \"service-ca-operator-5d6c75f84f-zmgn6\" (UID: \"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708660    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-config-volume\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708662    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2e1b4aee-0b22-46b0-80f7-362fd14d99fc-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c\" (UID: \"2e1b4aee-0b22-46b0-80f7-362fd14d99fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqphd\" (UniqueName: \"kubernetes.io/projected/fd434c96-8b02-486e-8486-4274ccb1b543-kube-api-access-rqphd\") pod \"network-check-source-7d88cb55d8-qxnvt\" (UID: \"fd434c96-8b02-486e-8486-4274ccb1b543\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-certificates\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pxqp\" (UniqueName: \"kubernetes.io/projected/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-kube-api-access-9pxqp\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2e1b4aee-0b22-46b0-80f7-362fd14d99fc-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c\" (UID: \"2e1b4aee-0b22-46b0-80f7-362fd14d99fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.708971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22ef61cc-b722-40e9-ad7e-4819d634238f-config\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4489r\" (UniqueName: \"kubernetes.io/projected/2e1b4aee-0b22-46b0-80f7-362fd14d99fc-kube-api-access-4489r\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c\" (UID: \"2e1b4aee-0b22-46b0-80f7-362fd14d99fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.709239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709053    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-default-certificate\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.710001 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709086    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzhb9\" (UniqueName: \"kubernetes.io/projected/3a511784-485f-4334-b266-e9e7dea58199-kube-api-access-pzhb9\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.710001 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.710001 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-stats-auth\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.710001 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709145    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2e1b4aee-0b22-46b0-80f7-362fd14d99fc-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c\" (UID: \"2e1b4aee-0b22-46b0-80f7-362fd14d99fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.710001 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a511784-485f-4334-b266-e9e7dea58199-service-ca-bundle\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.710001 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709439    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a511784-485f-4334-b266-e9e7dea58199-tmp\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.710001 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-certificates\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.710001 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.709822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22ef61cc-b722-40e9-ad7e-4819d634238f-serving-cert\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.710596    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.710618    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59cd4b4456-zprqx: secret "image-registry-tls" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.710682    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls podName:d7620cb9-8c18-4e77-a04d-df44b78842de nodeName:}" failed. No retries permitted until 2026-07-27 21:34:25.2106652 +0000 UTC m=+36.106474643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls") pod "image-registry-59cd4b4456-zprqx" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de") : secret "image-registry-tls" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.710691    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.710772    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls podName:2fd239d2-c68e-4edf-a18b-1e2888c14dcc nodeName:}" failed. No retries permitted until 2026-07-27 21:34:25.210754874 +0000 UTC m=+36.106564320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls") pod "dns-default-qdxcv" (UID: "2fd239d2-c68e-4edf-a18b-1e2888c14dcc") : secret "dns-default-metrics-tls" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.710832    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.710866    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls podName:a56677e1-f2c8-4000-b957-9cb3d83fc8a1 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:25.210855467 +0000 UTC m=+36.106664896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-g8gck" (UID: "a56677e1-f2c8-4000-b957-9cb3d83fc8a1") : secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.711028    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.711085    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls podName:4b70eb42-72b3-4069-83e0-f1cdb6e5740b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:25.211068997 +0000 UTC m=+36.106878442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls") pod "cluster-samples-operator-6748489656-mnk2z" (UID: "4b70eb42-72b3-4069-83e0-f1cdb6e5740b") : secret "samples-operator-tls" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.711131    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22ef61cc-b722-40e9-ad7e-4819d634238f-config\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.711207    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.711253    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:25.211240779 +0000 UTC m=+36.107050211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : secret "router-metrics-certs-default" not found
Jul 27 21:34:24.711624 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.711475    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29-config\") pod \"service-ca-operator-5d6c75f84f-zmgn6\" (UID: \"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.712518 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.712492    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22ef61cc-b722-40e9-ad7e-4819d634238f-trusted-ca\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.712609 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.712513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3a511784-485f-4334-b266-e9e7dea58199-snapshots\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.713061 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.713036    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a511784-485f-4334-b266-e9e7dea58199-service-ca-bundle\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.714215 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.713199    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0442a5bc-b902-4467-bfc2-508cc0aeab47-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:24.714215 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.713338    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:25.213320939 +0000 UTC m=+36.109130385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : configmap references non-existent config key: service-ca.crt
Jul 27 21:34:24.714215 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.713383    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-tmp-dir\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.714215 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.713624    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7620cb9-8c18-4e77-a04d-df44b78842de-ca-trust-extracted\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.714215 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.713671    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 21:34:24.714215 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:24.713718    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert podName:0442a5bc-b902-4467-bfc2-508cc0aeab47 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:25.21370397 +0000 UTC m=+36.109513404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dr8rd" (UID: "0442a5bc-b902-4467-bfc2-508cc0aeab47") : secret "networking-console-plugin-cert" not found
Jul 27 21:34:24.714947 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.714574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-trusted-ca\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.714947 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.714907    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:24.715372 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.715349    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a511784-485f-4334-b266-e9e7dea58199-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.715459 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.715379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-default-certificate\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.715862 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.715799    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-installation-pull-secrets\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.716509 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.716182    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d42297f-2c9c-4373-afc5-667b897f9c57-cert\") pod \"ingress-canary-8xsl4\" (UID: \"6d42297f-2c9c-4373-afc5-667b897f9c57\") " pod="openshift-ingress-canary/ingress-canary-8xsl4"
Jul 27 21:34:24.716509 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.716392    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2e1b4aee-0b22-46b0-80f7-362fd14d99fc-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c\" (UID: \"2e1b4aee-0b22-46b0-80f7-362fd14d99fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.716509 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.716472    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3a511784-485f-4334-b266-e9e7dea58199-serving-cert\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.716509 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.716503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22ef61cc-b722-40e9-ad7e-4819d634238f-serving-cert\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.716698 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.716524    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-stats-auth\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.717574 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.717522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29-serving-cert\") pod \"service-ca-operator-5d6c75f84f-zmgn6\" (UID: \"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.718474 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.718294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-image-registry-private-configuration\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.719503 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.719473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pxqp\" (UniqueName: \"kubernetes.io/projected/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-kube-api-access-9pxqp\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:24.720749 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.720704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b4cz\" (UniqueName: \"kubernetes.io/projected/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-kube-api-access-7b4cz\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:24.721533 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.721514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzhb9\" (UniqueName: \"kubernetes.io/projected/3a511784-485f-4334-b266-e9e7dea58199-kube-api-access-pzhb9\") pod \"insights-operator-5885fdb759-cpgfx\" (UID: \"3a511784-485f-4334-b266-e9e7dea58199\") " pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.721606 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.721512    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzfc4\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-kube-api-access-kzfc4\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.722608 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.722585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqphd\" (UniqueName: \"kubernetes.io/projected/fd434c96-8b02-486e-8486-4274ccb1b543-kube-api-access-rqphd\") pod \"network-check-source-7d88cb55d8-qxnvt\" (UID: \"fd434c96-8b02-486e-8486-4274ccb1b543\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt"
Jul 27 21:34:24.723265 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.723214    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4489r\" (UniqueName: \"kubernetes.io/projected/2e1b4aee-0b22-46b0-80f7-362fd14d99fc-kube-api-access-4489r\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c\" (UID: \"2e1b4aee-0b22-46b0-80f7-362fd14d99fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.724815 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.724797    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-bound-sa-token\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:24.725341 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.725304    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp4d2\" (UniqueName: \"kubernetes.io/projected/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-kube-api-access-xp4d2\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:24.726267 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.726227    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rk2l\" (UniqueName: \"kubernetes.io/projected/6d42297f-2c9c-4373-afc5-667b897f9c57-kube-api-access-2rk2l\") pod \"ingress-canary-8xsl4\" (UID: \"6d42297f-2c9c-4373-afc5-667b897f9c57\") " pod="openshift-ingress-canary/ingress-canary-8xsl4"
Jul 27 21:34:24.726462 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.726421    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhjw2\" (UniqueName: \"kubernetes.io/projected/22ef61cc-b722-40e9-ad7e-4819d634238f-kube-api-access-nhjw2\") pod \"console-operator-584dcb6b54-wzbsj\" (UID: \"22ef61cc-b722-40e9-ad7e-4819d634238f\") " pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.727044 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.727026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vp56\" (UniqueName: \"kubernetes.io/projected/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-kube-api-access-9vp56\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:24.727659 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.727639    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p59fm\" (UniqueName: \"kubernetes.io/projected/34756d4e-1ce5-4140-8585-fae29e941ab3-kube-api-access-p59fm\") pod \"volume-data-source-validator-549c6d54cf-vrdbh\" (UID: \"34756d4e-1ce5-4140-8585-fae29e941ab3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh"
Jul 27 21:34:24.728592 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.728563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chbhk\" (UniqueName: \"kubernetes.io/projected/8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29-kube-api-access-chbhk\") pod \"service-ca-operator-5d6c75f84f-zmgn6\" (UID: \"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.818674 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.818617    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh"
Jul 27 21:34:24.826719 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.826699    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"
Jul 27 21:34:24.833414 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.833392    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"
Jul 27 21:34:24.856166 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.856142    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-cpgfx"
Jul 27 21:34:24.864722 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.864699    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:24.877688 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.877668    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8xsl4"
Jul 27 21:34:24.884294 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:24.884276    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt"
Jul 27 21:34:25.213748 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:25.213694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:25.213912 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:25.213795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:25.213912 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:25.213826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:25.213912 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.213878    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 21:34:25.214069 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.213928    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:34:25.214069 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.213960    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:26.213937312 +0000 UTC m=+37.109746761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : secret "router-metrics-certs-default" not found
Jul 27 21:34:25.214069 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.213964    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 21:34:25.214069 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.213981    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls podName:2fd239d2-c68e-4edf-a18b-1e2888c14dcc nodeName:}" failed. No retries permitted until 2026-07-27 21:34:26.213968879 +0000 UTC m=+37.109778323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls") pod "dns-default-qdxcv" (UID: "2fd239d2-c68e-4edf-a18b-1e2888c14dcc") : secret "dns-default-metrics-tls" not found
Jul 27 21:34:25.214069 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.214026    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert podName:0442a5bc-b902-4467-bfc2-508cc0aeab47 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:26.214010296 +0000 UTC m=+37.109819730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dr8rd" (UID: "0442a5bc-b902-4467-bfc2-508cc0aeab47") : secret "networking-console-plugin-cert" not found
Jul 27 21:34:25.214069 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:25.213882    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:25.214069 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.214050    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:25.214120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:25.214156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.214163    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.214173    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59cd4b4456-zprqx: secret "image-registry-tls" not found
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:25.214181    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.214208    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls podName:a56677e1-f2c8-4000-b957-9cb3d83fc8a1 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:26.21418761 +0000 UTC m=+37.109997046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-g8gck" (UID: "a56677e1-f2c8-4000-b957-9cb3d83fc8a1") : secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.214236    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls podName:d7620cb9-8c18-4e77-a04d-df44b78842de nodeName:}" failed. No retries permitted until 2026-07-27 21:34:26.214224626 +0000 UTC m=+37.110034062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls") pod "image-registry-59cd4b4456-zprqx" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de") : secret "image-registry-tls" not found
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.214254    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.214264    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:26.214255715 +0000 UTC m=+37.110065144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : configmap references non-existent config key: service-ca.crt
Jul 27 21:34:25.214353 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:25.214346    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls podName:4b70eb42-72b3-4069-83e0-f1cdb6e5740b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:26.21433212 +0000 UTC m=+37.110141549 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls") pod "cluster-samples-operator-6748489656-mnk2z" (UID: "4b70eb42-72b3-4069-83e0-f1cdb6e5740b") : secret "samples-operator-tls" not found
Jul 27 21:34:26.075230 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.074868    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt"]
Jul 27 21:34:26.080065 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.079993    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-cpgfx"]
Jul 27 21:34:26.082117 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:26.080790    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd434c96_8b02_486e_8486_4274ccb1b543.slice/crio-4b411e1f3d924c3c206743ec976af198a1176c45e5b6ded3699bc69103cdd637 WatchSource:0}: Error finding container 4b411e1f3d924c3c206743ec976af198a1176c45e5b6ded3699bc69103cdd637: Status 404 returned error can't find the container with id 4b411e1f3d924c3c206743ec976af198a1176c45e5b6ded3699bc69103cdd637
Jul 27 21:34:26.084677 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:26.084208    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a511784_485f_4334_b266_e9e7dea58199.slice/crio-a200dfa4df16a8466305d9e62278493882cf217fae4bbc7592b81f53a61631bd WatchSource:0}: Error finding container a200dfa4df16a8466305d9e62278493882cf217fae4bbc7592b81f53a61631bd: Status 404 returned error can't find the container with id a200dfa4df16a8466305d9e62278493882cf217fae4bbc7592b81f53a61631bd
Jul 27 21:34:26.093202 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.093178    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-wzbsj"]
Jul 27 21:34:26.096613 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:26.096586    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ef61cc_b722_40e9_ad7e_4819d634238f.slice/crio-d77167f4aa21643966b289ff97699460ecc434fd250713b184e40ce7c761bac7 WatchSource:0}: Error finding container d77167f4aa21643966b289ff97699460ecc434fd250713b184e40ce7c761bac7: Status 404 returned error can't find the container with id d77167f4aa21643966b289ff97699460ecc434fd250713b184e40ce7c761bac7
Jul 27 21:34:26.099492 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.099472    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh"]
Jul 27 21:34:26.101578 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:26.101557    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34756d4e_1ce5_4140_8585_fae29e941ab3.slice/crio-b2765010e55c96b2cde1e818ff88b15b7a305ce86a2ce7c0f653d4db959b4b19 WatchSource:0}: Error finding container b2765010e55c96b2cde1e818ff88b15b7a305ce86a2ce7c0f653d4db959b4b19: Status 404 returned error can't find the container with id b2765010e55c96b2cde1e818ff88b15b7a305ce86a2ce7c0f653d4db959b4b19
Jul 27 21:34:26.107998 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.107864    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6"]
Jul 27 21:34:26.109564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.109543    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c"]
Jul 27 21:34:26.115808 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.115787    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8xsl4"]
Jul 27 21:34:26.118606 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:26.118581    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d42297f_2c9c_4373_afc5_667b897f9c57.slice/crio-8780dae937de47515b6d388916ca716cb86245f98b6b7282bd587b2d810d34cc WatchSource:0}: Error finding container 8780dae937de47515b6d388916ca716cb86245f98b6b7282bd587b2d810d34cc: Status 404 returned error can't find the container with id 8780dae937de47515b6d388916ca716cb86245f98b6b7282bd587b2d810d34cc
Jul 27 21:34:26.119255 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:26.119236    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e1b4aee_0b22_46b0_80f7_362fd14d99fc.slice/crio-76ef6c65d3d83a4458e71f43dd4d96290cde8011c6c59dc8262e1f064efa04e9 WatchSource:0}: Error finding container 76ef6c65d3d83a4458e71f43dd4d96290cde8011c6c59dc8262e1f064efa04e9: Status 404 returned error can't find the container with id 76ef6c65d3d83a4458e71f43dd4d96290cde8011c6c59dc8262e1f064efa04e9
Jul 27 21:34:26.120521 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:26.120398    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8d6e52_9b50_4c2e_a948_86a1bd2c7f29.slice/crio-8730385b8d133a615b99b975120786169f0ea879c1a776f4178124ebc9e436f1 WatchSource:0}: Error finding container 8730385b8d133a615b99b975120786169f0ea879c1a776f4178124ebc9e436f1: Status 404 returned error can't find the container with id 8730385b8d133a615b99b975120786169f0ea879c1a776f4178124ebc9e436f1
Jul 27 21:34:26.227409 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.227389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:26.227528 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.227424    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:26.227528 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.227452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:26.227528 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.227515    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:26.227679 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227546    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 21:34:26.227679 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227566    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59cd4b4456-zprqx: secret "image-registry-tls" not found
Jul 27 21:34:26.227679 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227557    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:28.227541269 +0000 UTC m=+39.123350698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : configmap references non-existent config key: service-ca.crt
Jul 27 21:34:26.227679 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227608    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 21:34:26.227679 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227609    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:34:26.227679 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227625    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls podName:d7620cb9-8c18-4e77-a04d-df44b78842de nodeName:}" failed. No retries permitted until 2026-07-27 21:34:28.227608047 +0000 UTC m=+39.123417478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls") pod "image-registry-59cd4b4456-zprqx" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de") : secret "image-registry-tls" not found
Jul 27 21:34:26.227679 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227651    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:28.22763797 +0000 UTC m=+39.123447417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : secret "router-metrics-certs-default" not found
Jul 27 21:34:26.227679 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227679    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls podName:4b70eb42-72b3-4069-83e0-f1cdb6e5740b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:28.227669539 +0000 UTC m=+39.123478973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls") pod "cluster-samples-operator-6748489656-mnk2z" (UID: "4b70eb42-72b3-4069-83e0-f1cdb6e5740b") : secret "samples-operator-tls" not found
Jul 27 21:34:26.228034 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.227713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:26.228034 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.227766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:26.228034 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227820    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:34:26.228034 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.227829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:26.228034 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227852    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls podName:2fd239d2-c68e-4edf-a18b-1e2888c14dcc nodeName:}" failed. No retries permitted until 2026-07-27 21:34:28.227842154 +0000 UTC m=+39.123651588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls") pod "dns-default-qdxcv" (UID: "2fd239d2-c68e-4edf-a18b-1e2888c14dcc") : secret "dns-default-metrics-tls" not found
Jul 27 21:34:26.228034 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227880    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:26.228034 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227888    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 21:34:26.228034 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227917    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls podName:a56677e1-f2c8-4000-b957-9cb3d83fc8a1 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:28.227906956 +0000 UTC m=+39.123716385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-g8gck" (UID: "a56677e1-f2c8-4000-b957-9cb3d83fc8a1") : secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:26.228034 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:26.227929    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert podName:0442a5bc-b902-4467-bfc2-508cc0aeab47 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:28.227923803 +0000 UTC m=+39.123733232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dr8rd" (UID: "0442a5bc-b902-4467-bfc2-508cc0aeab47") : secret "networking-console-plugin-cert" not found
Jul 27 21:34:26.811450 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.811411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt" event={"ID":"fd434c96-8b02-486e-8486-4274ccb1b543","Type":"ContainerStarted","Data":"4b411e1f3d924c3c206743ec976af198a1176c45e5b6ded3699bc69103cdd637"}
Jul 27 21:34:26.813681 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.813613    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6" event={"ID":"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29","Type":"ContainerStarted","Data":"8730385b8d133a615b99b975120786169f0ea879c1a776f4178124ebc9e436f1"}
Jul 27 21:34:26.815377 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.815305    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh" event={"ID":"34756d4e-1ce5-4140-8585-fae29e941ab3","Type":"ContainerStarted","Data":"b2765010e55c96b2cde1e818ff88b15b7a305ce86a2ce7c0f653d4db959b4b19"}
Jul 27 21:34:26.817129 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.817036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj" event={"ID":"22ef61cc-b722-40e9-ad7e-4819d634238f","Type":"ContainerStarted","Data":"d77167f4aa21643966b289ff97699460ecc434fd250713b184e40ce7c761bac7"}
Jul 27 21:34:26.820514 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.820432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8xsl4" event={"ID":"6d42297f-2c9c-4373-afc5-667b897f9c57","Type":"ContainerStarted","Data":"8780dae937de47515b6d388916ca716cb86245f98b6b7282bd587b2d810d34cc"}
Jul 27 21:34:26.822307 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.822282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-cpgfx" event={"ID":"3a511784-485f-4334-b266-e9e7dea58199","Type":"ContainerStarted","Data":"a200dfa4df16a8466305d9e62278493882cf217fae4bbc7592b81f53a61631bd"}
Jul 27 21:34:26.828726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.828698    2578 generic.go:356] "Generic (PLEG): container finished" podID="234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44" containerID="4a2c35d58ad7cff3fc8b892c77f4d1d4656ae21f4c158e988aefb80a565a5fad" exitCode=0
Jul 27 21:34:26.828834 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.828785    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtpwr" event={"ID":"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44","Type":"ContainerDied","Data":"4a2c35d58ad7cff3fc8b892c77f4d1d4656ae21f4c158e988aefb80a565a5fad"}
Jul 27 21:34:26.835806 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:26.834936    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c" event={"ID":"2e1b4aee-0b22-46b0-80f7-362fd14d99fc","Type":"ContainerStarted","Data":"76ef6c65d3d83a4458e71f43dd4d96290cde8011c6c59dc8262e1f064efa04e9"}
Jul 27 21:34:27.854080 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:27.854020    2578 generic.go:356] "Generic (PLEG): container finished" podID="234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44" containerID="81d02c4291a9b9e5a03374c4ce3ea35e75f9a5d0ec8b2b71f5341389b65871d9" exitCode=0
Jul 27 21:34:27.854631 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:27.854131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtpwr" event={"ID":"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44","Type":"ContainerDied","Data":"81d02c4291a9b9e5a03374c4ce3ea35e75f9a5d0ec8b2b71f5341389b65871d9"}
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.248544    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.248801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.248644    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.248859    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.248894    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls podName:2fd239d2-c68e-4edf-a18b-1e2888c14dcc nodeName:}" failed. No retries permitted until 2026-07-27 21:34:32.248872474 +0000 UTC m=+43.144681909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls") pod "dns-default-qdxcv" (UID: "2fd239d2-c68e-4edf-a18b-1e2888c14dcc") : secret "dns-default-metrics-tls" not found
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.248955    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.248961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.248979    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.248990    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249004    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert podName:0442a5bc-b902-4467-bfc2-508cc0aeab47 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:32.24898815 +0000 UTC m=+43.144797603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dr8rd" (UID: "0442a5bc-b902-4467-bfc2-508cc0aeab47") : secret "networking-console-plugin-cert" not found
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249030    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls podName:a56677e1-f2c8-4000-b957-9cb3d83fc8a1 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:32.249021588 +0000 UTC m=+43.144831023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-g8gck" (UID: "a56677e1-f2c8-4000-b957-9cb3d83fc8a1") : secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.249046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:28.249072 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249056    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:32.249045144 +0000 UTC m=+43.144854573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : configmap references non-existent config key: service-ca.crt
Jul 27 21:34:28.249893 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249099    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:34:28.249893 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.249116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:28.249893 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249127    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls podName:4b70eb42-72b3-4069-83e0-f1cdb6e5740b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:32.249118579 +0000 UTC m=+43.144928032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls") pod "cluster-samples-operator-6748489656-mnk2z" (UID: "4b70eb42-72b3-4069-83e0-f1cdb6e5740b") : secret "samples-operator-tls" not found
Jul 27 21:34:28.249893 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249170    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 21:34:28.249893 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249200    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:32.249190383 +0000 UTC m=+43.144999816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : secret "router-metrics-certs-default" not found
Jul 27 21:34:28.249893 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249238    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 21:34:28.249893 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249249    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59cd4b4456-zprqx: secret "image-registry-tls" not found
Jul 27 21:34:28.249893 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:28.249276    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls podName:d7620cb9-8c18-4e77-a04d-df44b78842de nodeName:}" failed. No retries permitted until 2026-07-27 21:34:32.249267305 +0000 UTC m=+43.145076740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls") pod "image-registry-59cd4b4456-zprqx" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de") : secret "image-registry-tls" not found
Jul 27 21:34:28.863429 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.863383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtpwr" event={"ID":"234c68e7-e353-4ed9-a6cf-0bbf6ecb9a44","Type":"ContainerStarted","Data":"24379346bef1e5dcf0e859874a2bba3b0c94a08066366f7377fdb4a33b7a861f"}
Jul 27 21:34:28.865517 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.865492    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh" event={"ID":"34756d4e-1ce5-4140-8585-fae29e941ab3","Type":"ContainerStarted","Data":"13f83678cc1bd7df1e5ff4f0585bc5552820e919a8f09a642b88b386616fda61"}
Jul 27 21:34:28.888417 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:28.887845    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xtpwr" podStartSLOduration=8.833376274 podStartE2EDuration="38.887816662s" totalImagesPullingTime="30.054440388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:28.885946251 +0000 UTC m=+39.781755702" watchObservedRunningTime="2026-07-27 21:34:28.887816662 +0000 UTC m=+39.783626114"
Jul 27 21:34:32.287205 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:32.287159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:32.287213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:32.287234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:32.287294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287319    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287329    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:32.287348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:32.287374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287391    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:40.287371364 +0000 UTC m=+51.183180797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : secret "router-metrics-certs-default" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287391    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287432    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287454    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls podName:2fd239d2-c68e-4edf-a18b-1e2888c14dcc nodeName:}" failed. No retries permitted until 2026-07-27 21:34:40.287440551 +0000 UTC m=+51.183249988 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls") pod "dns-default-qdxcv" (UID: "2fd239d2-c68e-4edf-a18b-1e2888c14dcc") : secret "dns-default-metrics-tls" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287471    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls podName:a56677e1-f2c8-4000-b957-9cb3d83fc8a1 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:40.28746154 +0000 UTC m=+51.183270971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-g8gck" (UID: "a56677e1-f2c8-4000-b957-9cb3d83fc8a1") : secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:32.287491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287516    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287531    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59cd4b4456-zprqx: secret "image-registry-tls" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287571    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls podName:d7620cb9-8c18-4e77-a04d-df44b78842de nodeName:}" failed. No retries permitted until 2026-07-27 21:34:40.287556035 +0000 UTC m=+51.183365470 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls") pod "image-registry-59cd4b4456-zprqx" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de") : secret "image-registry-tls" not found
Jul 27 21:34:32.287831 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287581    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:34:32.288506 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287588    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:40.287578961 +0000 UTC m=+51.183388390 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : configmap references non-existent config key: service-ca.crt
Jul 27 21:34:32.288506 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287639    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert podName:0442a5bc-b902-4467-bfc2-508cc0aeab47 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:40.287630303 +0000 UTC m=+51.183439732 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dr8rd" (UID: "0442a5bc-b902-4467-bfc2-508cc0aeab47") : secret "networking-console-plugin-cert" not found
Jul 27 21:34:32.288506 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:32.287649    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls podName:4b70eb42-72b3-4069-83e0-f1cdb6e5740b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:40.287643709 +0000 UTC m=+51.183453139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls") pod "cluster-samples-operator-6748489656-mnk2z" (UID: "4b70eb42-72b3-4069-83e0-f1cdb6e5740b") : secret "samples-operator-tls" not found
Jul 27 21:34:34.884077 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.884029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt" event={"ID":"fd434c96-8b02-486e-8486-4274ccb1b543","Type":"ContainerStarted","Data":"69997bb74a431bd4726561920b137cfd376b6ab11ae5b3f7f2e6bcbd8bec9714"}
Jul 27 21:34:34.885905 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.885874    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6" event={"ID":"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29","Type":"ContainerStarted","Data":"6d8bc6bb576b3a21cc5878a44ee1c46825ba6bb92da065c03fc5d69ce77486d3"}
Jul 27 21:34:34.887790 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.887767    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/0.log"
Jul 27 21:34:34.887912 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.887806    2578 generic.go:356] "Generic (PLEG): container finished" podID="22ef61cc-b722-40e9-ad7e-4819d634238f" containerID="d867e7fe6929a34e6be8c14e93b4d38a2b38e0f9e0ab9045a4cdab3b622c4369" exitCode=255
Jul 27 21:34:34.887912 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.887888    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj" event={"ID":"22ef61cc-b722-40e9-ad7e-4819d634238f","Type":"ContainerDied","Data":"d867e7fe6929a34e6be8c14e93b4d38a2b38e0f9e0ab9045a4cdab3b622c4369"}
Jul 27 21:34:34.888058 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.888041    2578 scope.go:117] "RemoveContainer" containerID="d867e7fe6929a34e6be8c14e93b4d38a2b38e0f9e0ab9045a4cdab3b622c4369"
Jul 27 21:34:34.889642 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.889605    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8xsl4" event={"ID":"6d42297f-2c9c-4373-afc5-667b897f9c57","Type":"ContainerStarted","Data":"ea0d9e639dfd993a06136e2eea814ff773d700f2e6e015ce8310d7fa60d5ad56"}
Jul 27 21:34:34.891233 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.891204    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-cpgfx" event={"ID":"3a511784-485f-4334-b266-e9e7dea58199","Type":"ContainerStarted","Data":"4bf1b2c25317c2d711cfa32d48164702ffde50127b4b9838b6e10dec451ec0d0"}
Jul 27 21:34:34.892693 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.892668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c" event={"ID":"2e1b4aee-0b22-46b0-80f7-362fd14d99fc","Type":"ContainerStarted","Data":"3f2817cc9329246a38be13175f2b51d73a18f7c8b157ab93f6b6c041b12e0e94"}
Jul 27 21:34:34.901098 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.901054    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-vrdbh" podStartSLOduration=22.787484383 podStartE2EDuration="24.90103899s" totalImagesPullingTime="2.113554607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:28.904884543 +0000 UTC m=+39.800693995" watchObservedRunningTime="2026-07-27 21:34:34.90103899 +0000 UTC m=+45.796848438"
Jul 27 21:34:34.901379 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.901352    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-qxnvt" podStartSLOduration=16.902603225 podStartE2EDuration="24.90134215s" totalImagesPullingTime="7.998738925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:34.900076826 +0000 UTC m=+45.795886271" watchObservedRunningTime="2026-07-27 21:34:34.90134215 +0000 UTC m=+45.797151582"
Jul 27 21:34:34.921942 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.921902    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c" podStartSLOduration=16.960256479999998 podStartE2EDuration="24.921887451s" totalImagesPullingTime="7.961630971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:34.92124193 +0000 UTC m=+45.817051382" watchObservedRunningTime="2026-07-27 21:34:34.921887451 +0000 UTC m=+45.817696884"
Jul 27 21:34:34.942948 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.942906    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-cpgfx" podStartSLOduration=16.946809028 podStartE2EDuration="24.942875519s" totalImagesPullingTime="7.996066491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:34.941578818 +0000 UTC m=+45.837388270" watchObservedRunningTime="2026-07-27 21:34:34.942875519 +0000 UTC m=+45.838684972"
Jul 27 21:34:34.978499 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.978214    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8xsl4" podStartSLOduration=3.00885882 podStartE2EDuration="10.978193462s" totalImagesPullingTime="7.969334642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:34.97742372 +0000 UTC m=+45.873233171" watchObservedRunningTime="2026-07-27 21:34:34.978193462 +0000 UTC m=+45.874002913"
Jul 27 21:34:34.995319 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:34.995272    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6" podStartSLOduration=17.035034947 podStartE2EDuration="24.995254244s" totalImagesPullingTime="7.960219297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:34.99477183 +0000 UTC m=+45.890581281" watchObservedRunningTime="2026-07-27 21:34:34.995254244 +0000 UTC m=+45.891063698"
Jul 27 21:34:35.680846 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.680803    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r5djc"]
Jul 27 21:34:35.706678 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.706654    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r5djc"]
Jul 27 21:34:35.706893 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.706810    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.709409 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.709379    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 21:34:35.709523 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.709497    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 21:34:35.709523 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.709508    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d6xkx\""
Jul 27 21:34:35.820458 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.820436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.820575 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.820490    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.820575 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.820539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eabb1707-799d-4ec3-a787-87acc0a7955b-crio-socket\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.820575 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.820570    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szxsm\" (UniqueName: \"kubernetes.io/projected/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-api-access-szxsm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.820681 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.820653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eabb1707-799d-4ec3-a787-87acc0a7955b-data-volume\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.896863 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.896844    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:34:35.897299 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.897286    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/0.log"
Jul 27 21:34:35.897345 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.897316    2578 generic.go:356] "Generic (PLEG): container finished" podID="22ef61cc-b722-40e9-ad7e-4819d634238f" containerID="b44e694ef57ee850c582e26d5c72bae0ad88e469761be9cf40ccda610ddf3aa3" exitCode=255
Jul 27 21:34:35.897408 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.897388    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj" event={"ID":"22ef61cc-b722-40e9-ad7e-4819d634238f","Type":"ContainerDied","Data":"b44e694ef57ee850c582e26d5c72bae0ad88e469761be9cf40ccda610ddf3aa3"}
Jul 27 21:34:35.897481 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.897439    2578 scope.go:117] "RemoveContainer" containerID="d867e7fe6929a34e6be8c14e93b4d38a2b38e0f9e0ab9045a4cdab3b622c4369"
Jul 27 21:34:35.897637 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.897616    2578 scope.go:117] "RemoveContainer" containerID="b44e694ef57ee850c582e26d5c72bae0ad88e469761be9cf40ccda610ddf3aa3"
Jul 27 21:34:35.897857 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:35.897824    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-wzbsj_openshift-console-operator(22ef61cc-b722-40e9-ad7e-4819d634238f)\"" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj" podUID="22ef61cc-b722-40e9-ad7e-4819d634238f"
Jul 27 21:34:35.921170 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.921145    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.921272 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.921196    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.921272 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.921215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eabb1707-799d-4ec3-a787-87acc0a7955b-crio-socket\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.921272 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.921235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szxsm\" (UniqueName: \"kubernetes.io/projected/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-api-access-szxsm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.921411 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.921285    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eabb1707-799d-4ec3-a787-87acc0a7955b-data-volume\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.921411 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:35.921289    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:34:35.921411 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.921293    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eabb1707-799d-4ec3-a787-87acc0a7955b-crio-socket\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.921411 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:35.921300    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:35.921411 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:35.921341    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm podName:eabb1707-799d-4ec3-a787-87acc0a7955b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:36.421324654 +0000 UTC m=+47.317134085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r5djc" (UID: "eabb1707-799d-4ec3-a787-87acc0a7955b") : configmap "kube-rbac-proxy" not found
Jul 27 21:34:35.921612 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:35.921427    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls podName:eabb1707-799d-4ec3-a787-87acc0a7955b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:36.421412944 +0000 UTC m=+47.317222376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5djc" (UID: "eabb1707-799d-4ec3-a787-87acc0a7955b") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:35.921612 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.921577    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eabb1707-799d-4ec3-a787-87acc0a7955b-data-volume\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:35.934094 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:35.934043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szxsm\" (UniqueName: \"kubernetes.io/projected/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-api-access-szxsm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:36.426153 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:36.426122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:36.426302 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:36.426241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:36.426302 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:36.426244    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:34:36.426302 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:36.426301    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm podName:eabb1707-799d-4ec3-a787-87acc0a7955b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:37.426286427 +0000 UTC m=+48.322095860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r5djc" (UID: "eabb1707-799d-4ec3-a787-87acc0a7955b") : configmap "kube-rbac-proxy" not found
Jul 27 21:34:36.426462 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:36.426345    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:36.426462 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:36.426401    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls podName:eabb1707-799d-4ec3-a787-87acc0a7955b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:37.426386876 +0000 UTC m=+48.322196306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5djc" (UID: "eabb1707-799d-4ec3-a787-87acc0a7955b") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:36.902269 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:36.902243    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:34:36.902634 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:36.902548    2578 scope.go:117] "RemoveContainer" containerID="b44e694ef57ee850c582e26d5c72bae0ad88e469761be9cf40ccda610ddf3aa3"
Jul 27 21:34:36.902713 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:36.902693    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-wzbsj_openshift-console-operator(22ef61cc-b722-40e9-ad7e-4819d634238f)\"" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj" podUID="22ef61cc-b722-40e9-ad7e-4819d634238f"
Jul 27 21:34:37.082192 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:37.082166    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ftzbq_c9f9f3fc-17d0-4b1d-8a50-01757a4f934c/dns-node-resolver/0.log"
Jul 27 21:34:37.437171 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:37.437143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:37.437325 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:37.437196    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:37.437325 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:37.437254    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:34:37.437325 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:37.437298    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:37.437452 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:37.437302    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm podName:eabb1707-799d-4ec3-a787-87acc0a7955b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:39.437289636 +0000 UTC m=+50.333099065 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r5djc" (UID: "eabb1707-799d-4ec3-a787-87acc0a7955b") : configmap "kube-rbac-proxy" not found
Jul 27 21:34:37.437452 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:37.437372    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls podName:eabb1707-799d-4ec3-a787-87acc0a7955b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:39.4373516 +0000 UTC m=+50.333161030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5djc" (UID: "eabb1707-799d-4ec3-a787-87acc0a7955b") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:38.081084 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:38.081057    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-22g6x_d32a5315-4ac2-4d35-b133-854854da98c7/node-ca/0.log"
Jul 27 21:34:38.881943 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:38.881882    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8xsl4_6d42297f-2c9c-4373-afc5-667b897f9c57/serve-healthcheck-canary/0.log"
Jul 27 21:34:39.453936 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:39.453898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:39.454322 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:39.453969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:39.454322 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:39.454059    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:34:39.454322 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:39.454103    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:39.454322 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:39.454131    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm podName:eabb1707-799d-4ec3-a787-87acc0a7955b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:43.454113163 +0000 UTC m=+54.349922596 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r5djc" (UID: "eabb1707-799d-4ec3-a787-87acc0a7955b") : configmap "kube-rbac-proxy" not found
Jul 27 21:34:39.454322 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:39.454158    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls podName:eabb1707-799d-4ec3-a787-87acc0a7955b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:43.454145474 +0000 UTC m=+54.349954903 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5djc" (UID: "eabb1707-799d-4ec3-a787-87acc0a7955b") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:40.361656 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:40.361617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:40.361656 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:40.361657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:40.361940 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:40.361674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:40.361940 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.361802    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:34:40.361940 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.361813    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 21:34:40.361940 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:40.361826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:40.361940 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.361837    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59cd4b4456-zprqx: secret "image-registry-tls" not found
Jul 27 21:34:40.361940 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.361862    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls podName:4b70eb42-72b3-4069-83e0-f1cdb6e5740b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:56.361848192 +0000 UTC m=+67.257657621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls") pod "cluster-samples-operator-6748489656-mnk2z" (UID: "4b70eb42-72b3-4069-83e0-f1cdb6e5740b") : secret "samples-operator-tls" not found
Jul 27 21:34:40.361940 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.361910    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:56.361883464 +0000 UTC m=+67.257692897 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : configmap references non-existent config key: service-ca.crt
Jul 27 21:34:40.361940 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.361923    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 21:34:40.361940 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.361942    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls podName:d7620cb9-8c18-4e77-a04d-df44b78842de nodeName:}" failed. No retries permitted until 2026-07-27 21:34:56.361932331 +0000 UTC m=+67.257741760 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls") pod "image-registry-59cd4b4456-zprqx" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de") : secret "image-registry-tls" not found
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.361976    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs podName:4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f nodeName:}" failed. No retries permitted until 2026-07-27 21:34:56.361957443 +0000 UTC m=+67.257766873 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs") pod "router-default-64884fdbff-j7wtj" (UID: "4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f") : secret "router-metrics-certs-default" not found
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:40.362042    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:40.362076    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:40.362141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.362158    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.362197    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.362201    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls podName:2fd239d2-c68e-4edf-a18b-1e2888c14dcc nodeName:}" failed. No retries permitted until 2026-07-27 21:34:56.362194067 +0000 UTC m=+67.258003496 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls") pod "dns-default-qdxcv" (UID: "2fd239d2-c68e-4edf-a18b-1e2888c14dcc") : secret "dns-default-metrics-tls" not found
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.362243    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls podName:a56677e1-f2c8-4000-b957-9cb3d83fc8a1 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:56.362233705 +0000 UTC m=+67.258043134 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-g8gck" (UID: "a56677e1-f2c8-4000-b957-9cb3d83fc8a1") : secret "cluster-monitoring-operator-tls" not found
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.362275    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 21:34:40.362390 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:40.362329    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert podName:0442a5bc-b902-4467-bfc2-508cc0aeab47 nodeName:}" failed. No retries permitted until 2026-07-27 21:34:56.362314833 +0000 UTC m=+67.258124301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dr8rd" (UID: "0442a5bc-b902-4467-bfc2-508cc0aeab47") : secret "networking-console-plugin-cert" not found
Jul 27 21:34:43.488281 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:43.488246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:43.488887 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:43.488310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:43.488887 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:43.488421    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:43.488887 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:43.488497    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls podName:eabb1707-799d-4ec3-a787-87acc0a7955b nodeName:}" failed. No retries permitted until 2026-07-27 21:34:51.488480114 +0000 UTC m=+62.384289542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5djc" (UID: "eabb1707-799d-4ec3-a787-87acc0a7955b") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:34:43.489012 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:43.488941    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eabb1707-799d-4ec3-a787-87acc0a7955b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:44.865422 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:44.865391    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:44.865837 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:44.865445    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:44.865837 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:44.865800    2578 scope.go:117] "RemoveContainer" containerID="b44e694ef57ee850c582e26d5c72bae0ad88e469761be9cf40ccda610ddf3aa3"
Jul 27 21:34:44.866003 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:34:44.865982    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-wzbsj_openshift-console-operator(22ef61cc-b722-40e9-ad7e-4819d634238f)\"" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj" podUID="22ef61cc-b722-40e9-ad7e-4819d634238f"
Jul 27 21:34:50.806194 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:50.806169    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4p2b"
Jul 27 21:34:51.555231 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:51.555201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:51.557473 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:51.557448    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eabb1707-799d-4ec3-a787-87acc0a7955b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5djc\" (UID: \"eabb1707-799d-4ec3-a787-87acc0a7955b\") " pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:51.618687 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:51.618663    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d6xkx\""
Jul 27 21:34:51.627089 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:51.627069    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r5djc"
Jul 27 21:34:51.781234 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:51.781196    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r5djc"]
Jul 27 21:34:51.785633 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:51.785609    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeabb1707_799d_4ec3_a787_87acc0a7955b.slice/crio-61902fd129dc917b8c56f8ac593946a9d245d4d7b7dfa83c60fe538b4cc1fb6b WatchSource:0}: Error finding container 61902fd129dc917b8c56f8ac593946a9d245d4d7b7dfa83c60fe538b4cc1fb6b: Status 404 returned error can't find the container with id 61902fd129dc917b8c56f8ac593946a9d245d4d7b7dfa83c60fe538b4cc1fb6b
Jul 27 21:34:51.952751 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:51.952710    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5djc" event={"ID":"eabb1707-799d-4ec3-a787-87acc0a7955b","Type":"ContainerStarted","Data":"61902fd129dc917b8c56f8ac593946a9d245d4d7b7dfa83c60fe538b4cc1fb6b"}
Jul 27 21:34:55.391416 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.391378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:55.391981 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.391505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:55.391981 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.391545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:55.394710 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.394442    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 21:34:55.394710 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.394511    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 21:34:55.394914 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.394813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zks2p\" (UniqueName: \"kubernetes.io/projected/77f6e728-ad76-4515-a9d5-b13d3b72967e-kube-api-access-zks2p\") pod \"network-check-target-4g95m\" (UID: \"77f6e728-ad76-4515-a9d5-b13d3b72967e\") " pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:55.404683 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.404659    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3b83b76-20e7-43e7-85bd-885ab2b22ce8-original-pull-secret\") pod \"global-pull-secret-syncer-2q7dv\" (UID: \"d3b83b76-20e7-43e7-85bd-885ab2b22ce8\") " pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:55.405442 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.405416    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ea190da-d6af-4616-93a3-4ad59b61282a-metrics-certs\") pod \"network-metrics-daemon-wtmrw\" (UID: \"4ea190da-d6af-4616-93a3-4ad59b61282a\") " pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:55.518399 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.518363    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcmmw\""
Jul 27 21:34:55.520932 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.520904    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xnkh\""
Jul 27 21:34:55.522987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.522963    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:55.524313 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.524295    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q7dv"
Jul 27 21:34:55.528416 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.528398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wtmrw"
Jul 27 21:34:55.599696 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:55.599665    2578 scope.go:117] "RemoveContainer" containerID="b44e694ef57ee850c582e26d5c72bae0ad88e469761be9cf40ccda610ddf3aa3"
Jul 27 21:34:56.400891 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.400850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:56.401304 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.400905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:56.401304 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.400944    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:56.401304 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.400994    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:56.401304 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.401158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:56.401304 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.401203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:56.401304 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.401265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:56.402272 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.402215    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-service-ca-bundle\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:56.404136 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.404104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fd239d2-c68e-4edf-a18b-1e2888c14dcc-metrics-tls\") pod \"dns-default-qdxcv\" (UID: \"2fd239d2-c68e-4edf-a18b-1e2888c14dcc\") " pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:56.404274 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.404161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442a5bc-b902-4467-bfc2-508cc0aeab47-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dr8rd\" (UID: \"0442a5bc-b902-4467-bfc2-508cc0aeab47\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:56.404409 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.404384    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"image-registry-59cd4b4456-zprqx\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") " pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:56.404519 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.404383    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a56677e1-f2c8-4000-b957-9cb3d83fc8a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-g8gck\" (UID: \"a56677e1-f2c8-4000-b957-9cb3d83fc8a1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:56.404988 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.404945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b70eb42-72b3-4069-83e0-f1cdb6e5740b-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mnk2z\" (UID: \"4b70eb42-72b3-4069-83e0-f1cdb6e5740b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:56.405791 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.405771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f-metrics-certs\") pod \"router-default-64884fdbff-j7wtj\" (UID: \"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f\") " pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:56.579591 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.579564    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6vht4\""
Jul 27 21:34:56.586997 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.586827    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:56.589507 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.589459    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zzpc9\""
Jul 27 21:34:56.597862 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.597806    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"
Jul 27 21:34:56.604631 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.604437    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z2vzc\""
Jul 27 21:34:56.612956 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.612784    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:56.642362 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.642333    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kb678\""
Jul 27 21:34:56.649088 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.649065    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pfmkx\""
Jul 27 21:34:56.650158 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.649943    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:34:56.656913 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.656888    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"
Jul 27 21:34:56.674195 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.673987    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6w5sb\""
Jul 27 21:34:56.682374 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.682343    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"
Jul 27 21:34:56.711651 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.711585    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2q7dv"]
Jul 27 21:34:56.715188 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:56.715154    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b83b76_20e7_43e7_85bd_885ab2b22ce8.slice/crio-02f86834ef36272ec3448487ec8df69856cd4ff1f42bb74e10659d51d4df7e73 WatchSource:0}: Error finding container 02f86834ef36272ec3448487ec8df69856cd4ff1f42bb74e10659d51d4df7e73: Status 404 returned error can't find the container with id 02f86834ef36272ec3448487ec8df69856cd4ff1f42bb74e10659d51d4df7e73
Jul 27 21:34:56.725318 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.725295    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wtmrw"]
Jul 27 21:34:56.728613 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:56.728589    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea190da_d6af_4616_93a3_4ad59b61282a.slice/crio-1cdae84907a769d2b84f088ce349a0c27f0366f945ef8735db91166cb85383f9 WatchSource:0}: Error finding container 1cdae84907a769d2b84f088ce349a0c27f0366f945ef8735db91166cb85383f9: Status 404 returned error can't find the container with id 1cdae84907a769d2b84f088ce349a0c27f0366f945ef8735db91166cb85383f9
Jul 27 21:34:56.746987 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.746957    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4g95m"]
Jul 27 21:34:56.755817 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:56.755792    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f6e728_ad76_4515_a9d5_b13d3b72967e.slice/crio-64ca7ae09e60c789591fe2643ef100a641a5196d4aaf8af86fda609866ba26cb WatchSource:0}: Error finding container 64ca7ae09e60c789591fe2643ef100a641a5196d4aaf8af86fda609866ba26cb: Status 404 returned error can't find the container with id 64ca7ae09e60c789591fe2643ef100a641a5196d4aaf8af86fda609866ba26cb
Jul 27 21:34:56.972713 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.972680    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2q7dv" event={"ID":"d3b83b76-20e7-43e7-85bd-885ab2b22ce8","Type":"ContainerStarted","Data":"02f86834ef36272ec3448487ec8df69856cd4ff1f42bb74e10659d51d4df7e73"}
Jul 27 21:34:56.975160 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.975137    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:34:56.975285 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.975240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj" event={"ID":"22ef61cc-b722-40e9-ad7e-4819d634238f","Type":"ContainerStarted","Data":"a8b7ab453ca5700e2063a5f85acf41ba4460c6ac3e23486a0794b578e8515fb5"}
Jul 27 21:34:56.975569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.975553    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:56.977983 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.977952    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5djc" event={"ID":"eabb1707-799d-4ec3-a787-87acc0a7955b","Type":"ContainerStarted","Data":"243bf70cafa187664316ed9396d07beb0e5431b0db69faf29f451d3adfbaa8ae"}
Jul 27 21:34:56.979667 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.979643    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4g95m" event={"ID":"77f6e728-ad76-4515-a9d5-b13d3b72967e","Type":"ContainerStarted","Data":"64af3c81d439d59d82103b8bd7c378a76d2748a5d14bf0d929d35bcebdf1797b"}
Jul 27 21:34:56.979761 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.979685    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4g95m" event={"ID":"77f6e728-ad76-4515-a9d5-b13d3b72967e","Type":"ContainerStarted","Data":"64ca7ae09e60c789591fe2643ef100a641a5196d4aaf8af86fda609866ba26cb"}
Jul 27 21:34:56.980105 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.980090    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:34:56.981286 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:56.981269    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wtmrw" event={"ID":"4ea190da-d6af-4616-93a3-4ad59b61282a","Type":"ContainerStarted","Data":"1cdae84907a769d2b84f088ce349a0c27f0366f945ef8735db91166cb85383f9"}
Jul 27 21:34:57.002167 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.001788    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59cd4b4456-zprqx"]
Jul 27 21:34:57.009378 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:57.009339    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7620cb9_8c18_4e77_a04d_df44b78842de.slice/crio-6eb88b43a74502fdddad9daeba79724167c04e6866c4943430370fcafdc8c785 WatchSource:0}: Error finding container 6eb88b43a74502fdddad9daeba79724167c04e6866c4943430370fcafdc8c785: Status 404 returned error can't find the container with id 6eb88b43a74502fdddad9daeba79724167c04e6866c4943430370fcafdc8c785
Jul 27 21:34:57.025378 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.025328    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj" podStartSLOduration=39.041615964 podStartE2EDuration="47.02530714s" totalImagesPullingTime="7.983691176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:57.021344397 +0000 UTC m=+67.917153848" watchObservedRunningTime="2026-07-27 21:34:57.02530714 +0000 UTC m=+67.921116592"
Jul 27 21:34:57.028579 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.028557    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd"]
Jul 27 21:34:57.029522 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:57.029496    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0442a5bc_b902_4467_bfc2_508cc0aeab47.slice/crio-697e97f866d48d24736beb6562f6f5e6380586af2fd741cafe133402990e3ba3 WatchSource:0}: Error finding container 697e97f866d48d24736beb6562f6f5e6380586af2fd741cafe133402990e3ba3: Status 404 returned error can't find the container with id 697e97f866d48d24736beb6562f6f5e6380586af2fd741cafe133402990e3ba3
Jul 27 21:34:57.066691 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.066581    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4g95m" podStartSLOduration=67.066560048 podStartE2EDuration="1m7.066560048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:57.065990159 +0000 UTC m=+67.961799603" watchObservedRunningTime="2026-07-27 21:34:57.066560048 +0000 UTC m=+67.962369501"
Jul 27 21:34:57.080076 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.080053    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64884fdbff-j7wtj"]
Jul 27 21:34:57.080700 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:34:57.080678    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb0a7c8_bc6b_449a_acbe_4ed3bf0c955f.slice/crio-fe299341407ce5f41ece9f320f66ef9dce8f98525ce5914f881b4d6e4a0d3aad WatchSource:0}: Error finding container fe299341407ce5f41ece9f320f66ef9dce8f98525ce5914f881b4d6e4a0d3aad: Status 404 returned error can't find the container with id fe299341407ce5f41ece9f320f66ef9dce8f98525ce5914f881b4d6e4a0d3aad
Jul 27 21:34:57.201776 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.201679    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck"]
Jul 27 21:34:57.250809 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.250718    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qdxcv"]
Jul 27 21:34:57.253476 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.253414    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z"]
Jul 27 21:34:57.806107 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.806079    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-wzbsj"
Jul 27 21:34:57.991531 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.991494    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd" event={"ID":"0442a5bc-b902-4467-bfc2-508cc0aeab47","Type":"ContainerStarted","Data":"697e97f866d48d24736beb6562f6f5e6380586af2fd741cafe133402990e3ba3"}
Jul 27 21:34:57.994900 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.994061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx" event={"ID":"d7620cb9-8c18-4e77-a04d-df44b78842de","Type":"ContainerStarted","Data":"9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e"}
Jul 27 21:34:57.994900 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.994112    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx" event={"ID":"d7620cb9-8c18-4e77-a04d-df44b78842de","Type":"ContainerStarted","Data":"6eb88b43a74502fdddad9daeba79724167c04e6866c4943430370fcafdc8c785"}
Jul 27 21:34:57.994900 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.994856    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:34:57.997424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:57.997345    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qdxcv" event={"ID":"2fd239d2-c68e-4edf-a18b-1e2888c14dcc","Type":"ContainerStarted","Data":"c9a2dcb51019ef9c99a3b02bb2ca2e48f56a9361039ca935483204b0861bc52e"}
Jul 27 21:34:58.006432 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:58.005234    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck" event={"ID":"a56677e1-f2c8-4000-b957-9cb3d83fc8a1","Type":"ContainerStarted","Data":"903c2c22ee2db835d568a4c9e6ae3e41dc1308631fc600b36d383de49a7de08a"}
Jul 27 21:34:58.007982 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:58.007873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64884fdbff-j7wtj" event={"ID":"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f","Type":"ContainerStarted","Data":"4f0697ddc474fe392895011025a80b7e8bc9dd59da596a7cf6065a767ef48888"}
Jul 27 21:34:58.007982 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:58.007928    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64884fdbff-j7wtj" event={"ID":"4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f","Type":"ContainerStarted","Data":"fe299341407ce5f41ece9f320f66ef9dce8f98525ce5914f881b4d6e4a0d3aad"}
Jul 27 21:34:58.012270 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:58.011868    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z" event={"ID":"4b70eb42-72b3-4069-83e0-f1cdb6e5740b","Type":"ContainerStarted","Data":"4839adfc5ed265a0bd1dbf73baf710a68baafbb83b1aeeefaf8cb65321e310d4"}
Jul 27 21:34:58.034013 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:58.033954    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx" podStartSLOduration=68.033934939 podStartE2EDuration="1m8.033934939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:58.031076082 +0000 UTC m=+68.926885557" watchObservedRunningTime="2026-07-27 21:34:58.033934939 +0000 UTC m=+68.929744394"
Jul 27 21:34:58.062679 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:58.062551    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64884fdbff-j7wtj" podStartSLOduration=48.062531284 podStartE2EDuration="48.062531284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:34:58.061452125 +0000 UTC m=+68.957261577" watchObservedRunningTime="2026-07-27 21:34:58.062531284 +0000 UTC m=+68.958340739"
Jul 27 21:34:58.613803 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:58.613773    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:58.616939 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:58.616912    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:59.026997 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:59.026967    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:34:59.028495 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:34:59.028474    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64884fdbff-j7wtj"
Jul 27 21:35:03.182642 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.181544    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"]
Jul 27 21:35:03.199380 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.199277    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.200516 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.200477    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"]
Jul 27 21:35:03.204392 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.204365    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 21:35:03.204762 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.204721    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 21:35:03.211726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.211690    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 21:35:03.212482 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.212277    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 27 21:35:03.269080 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.267921    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-6b4dx"]
Jul 27 21:35:03.284078 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.283987    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-6b4dx"
Jul 27 21:35:03.285826 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.285800    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-6b4dx"]
Jul 27 21:35:03.288486 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.288461    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-l6bvf\""
Jul 27 21:35:03.288628 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.288461    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 27 21:35:03.289208 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.289039    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 27 21:35:03.363256 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.363218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e1f716b5-56a7-45a9-941a-e6ac4e380bc8-klusterlet-config\") pod \"klusterlet-addon-workmgr-d6df559b5-x4vrh\" (UID: \"e1f716b5-56a7-45a9-941a-e6ac4e380bc8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.363440 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.363259    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvzcl\" (UniqueName: \"kubernetes.io/projected/e1f716b5-56a7-45a9-941a-e6ac4e380bc8-kube-api-access-gvzcl\") pod \"klusterlet-addon-workmgr-d6df559b5-x4vrh\" (UID: \"e1f716b5-56a7-45a9-941a-e6ac4e380bc8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.363440 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.363337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1f716b5-56a7-45a9-941a-e6ac4e380bc8-tmp\") pod \"klusterlet-addon-workmgr-d6df559b5-x4vrh\" (UID: \"e1f716b5-56a7-45a9-941a-e6ac4e380bc8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.464645 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.464571    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmz5t\" (UniqueName: \"kubernetes.io/projected/503f8bfb-1452-4193-96c3-5d6dc36bc410-kube-api-access-nmz5t\") pod \"downloads-7c97dc4555-6b4dx\" (UID: \"503f8bfb-1452-4193-96c3-5d6dc36bc410\") " pod="openshift-console/downloads-7c97dc4555-6b4dx"
Jul 27 21:35:03.464827 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.464642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e1f716b5-56a7-45a9-941a-e6ac4e380bc8-klusterlet-config\") pod \"klusterlet-addon-workmgr-d6df559b5-x4vrh\" (UID: \"e1f716b5-56a7-45a9-941a-e6ac4e380bc8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.464827 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.464669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvzcl\" (UniqueName: \"kubernetes.io/projected/e1f716b5-56a7-45a9-941a-e6ac4e380bc8-kube-api-access-gvzcl\") pod \"klusterlet-addon-workmgr-d6df559b5-x4vrh\" (UID: \"e1f716b5-56a7-45a9-941a-e6ac4e380bc8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.464827 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.464718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1f716b5-56a7-45a9-941a-e6ac4e380bc8-tmp\") pod \"klusterlet-addon-workmgr-d6df559b5-x4vrh\" (UID: \"e1f716b5-56a7-45a9-941a-e6ac4e380bc8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.465078 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.465060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1f716b5-56a7-45a9-941a-e6ac4e380bc8-tmp\") pod \"klusterlet-addon-workmgr-d6df559b5-x4vrh\" (UID: \"e1f716b5-56a7-45a9-941a-e6ac4e380bc8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.467768 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.467718    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e1f716b5-56a7-45a9-941a-e6ac4e380bc8-klusterlet-config\") pod \"klusterlet-addon-workmgr-d6df559b5-x4vrh\" (UID: \"e1f716b5-56a7-45a9-941a-e6ac4e380bc8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.481776 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.481746    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvzcl\" (UniqueName: \"kubernetes.io/projected/e1f716b5-56a7-45a9-941a-e6ac4e380bc8-kube-api-access-gvzcl\") pod \"klusterlet-addon-workmgr-d6df559b5-x4vrh\" (UID: \"e1f716b5-56a7-45a9-941a-e6ac4e380bc8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.513076 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.513049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:03.565799 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.565757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmz5t\" (UniqueName: \"kubernetes.io/projected/503f8bfb-1452-4193-96c3-5d6dc36bc410-kube-api-access-nmz5t\") pod \"downloads-7c97dc4555-6b4dx\" (UID: \"503f8bfb-1452-4193-96c3-5d6dc36bc410\") " pod="openshift-console/downloads-7c97dc4555-6b4dx"
Jul 27 21:35:03.579332 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.579306    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmz5t\" (UniqueName: \"kubernetes.io/projected/503f8bfb-1452-4193-96c3-5d6dc36bc410-kube-api-access-nmz5t\") pod \"downloads-7c97dc4555-6b4dx\" (UID: \"503f8bfb-1452-4193-96c3-5d6dc36bc410\") " pod="openshift-console/downloads-7c97dc4555-6b4dx"
Jul 27 21:35:03.596546 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:03.596518    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-6b4dx"
Jul 27 21:35:06.217024 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:06.216562    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-6b4dx"]
Jul 27 21:35:06.239756 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:06.239714    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"]
Jul 27 21:35:07.079052 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.079015    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z" event={"ID":"4b70eb42-72b3-4069-83e0-f1cdb6e5740b","Type":"ContainerStarted","Data":"bc58d6a620b20e6239944c46207430673595b174fe018fe2d87bf4b2380d0b2a"}
Jul 27 21:35:07.079224 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.079083    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z" event={"ID":"4b70eb42-72b3-4069-83e0-f1cdb6e5740b","Type":"ContainerStarted","Data":"543a664cfdf57ff79253456fbc96a4872d3e41ae110b841937847ff75e0cd89e"}
Jul 27 21:35:07.082761 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.082260    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wtmrw" event={"ID":"4ea190da-d6af-4616-93a3-4ad59b61282a","Type":"ContainerStarted","Data":"713b5a99a43f0eb522031a0de12e8ed36c1485f08d12769a4b91092122eeff59"}
Jul 27 21:35:07.082761 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.082315    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wtmrw" event={"ID":"4ea190da-d6af-4616-93a3-4ad59b61282a","Type":"ContainerStarted","Data":"33cfce4b0b799694a3a9fd8e80cfaa7cbe94cc98220e3e460f8aded3566e3966"}
Jul 27 21:35:07.086561 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.085969    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qdxcv" event={"ID":"2fd239d2-c68e-4edf-a18b-1e2888c14dcc","Type":"ContainerStarted","Data":"206641e28a1e1a88c37993ce4f3fbe12678cd88e2765a97cdbf69c46e167c94a"}
Jul 27 21:35:07.086561 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.086014    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qdxcv" event={"ID":"2fd239d2-c68e-4edf-a18b-1e2888c14dcc","Type":"ContainerStarted","Data":"8fb38b62f643f045535162aed328d4e38cd642a7f0df7f4425ff7954f86d521e"}
Jul 27 21:35:07.086561 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.086507    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:35:07.088170 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.088107    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-6b4dx" event={"ID":"503f8bfb-1452-4193-96c3-5d6dc36bc410","Type":"ContainerStarted","Data":"4d34b85c084505c270b6cd5e6be7fd8c74b48291633a853022caa8f7e8ceca20"}
Jul 27 21:35:07.089500 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.089431    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh" event={"ID":"e1f716b5-56a7-45a9-941a-e6ac4e380bc8","Type":"ContainerStarted","Data":"1136e2518d99cb4e5fdedb305e251b33332816bf08db4d3ad704f4562a70941a"}
Jul 27 21:35:07.092882 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.092815    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5djc" event={"ID":"eabb1707-799d-4ec3-a787-87acc0a7955b","Type":"ContainerStarted","Data":"945e2022723462f762a125958a8140d1adc008aa6a442cbcfef4a538ec2e35d5"}
Jul 27 21:35:07.096258 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.095434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2q7dv" event={"ID":"d3b83b76-20e7-43e7-85bd-885ab2b22ce8","Type":"ContainerStarted","Data":"0227179b6b1c23cba6ec91e6e585bb24425008b259641769e3d44a547269775e"}
Jul 27 21:35:07.100535 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.100489    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mnk2z" podStartSLOduration=48.424494392 podStartE2EDuration="57.10047255s" totalImagesPullingTime="8.675978158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:07.09843786 +0000 UTC m=+77.994247323" watchObservedRunningTime="2026-07-27 21:35:07.10047255 +0000 UTC m=+77.996281984"
Jul 27 21:35:07.100649 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.100571    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck" event={"ID":"a56677e1-f2c8-4000-b957-9cb3d83fc8a1","Type":"ContainerStarted","Data":"85e2498aa31dc72353bc025c9e07eb8ded410652f61f7b0690ce3d3786e81337"}
Jul 27 21:35:07.104403 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.104359    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd" event={"ID":"0442a5bc-b902-4467-bfc2-508cc0aeab47","Type":"ContainerStarted","Data":"19507d4bafce7bed17f0492089757028ff5e92d6b5136d0c904fd8be96fafa26"}
Jul 27 21:35:07.115978 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.115828    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2q7dv" podStartSLOduration=67.869562188 podStartE2EDuration="1m17.115770079s" totalImagesPullingTime="9.246207891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:07.114586311 +0000 UTC m=+78.010395763" watchObservedRunningTime="2026-07-27 21:35:07.115770079 +0000 UTC m=+78.011579522"
Jul 27 21:35:07.135530 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.135487    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qdxcv" podStartSLOduration=34.503824746 podStartE2EDuration="43.135472694s" totalImagesPullingTime="8.631647948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:07.135291619 +0000 UTC m=+78.031101074" watchObservedRunningTime="2026-07-27 21:35:07.135472694 +0000 UTC m=+78.031282144"
Jul 27 21:35:07.155246 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.155089    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wtmrw" podStartSLOduration=69.221341194 podStartE2EDuration="1m17.155070115s" totalImagesPullingTime="7.933728921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:07.153170205 +0000 UTC m=+78.048979658" watchObservedRunningTime="2026-07-27 21:35:07.155070115 +0000 UTC m=+78.050879568"
Jul 27 21:35:07.177443 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.177291    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-g8gck" podStartSLOduration=48.397805907 podStartE2EDuration="57.177271428s" totalImagesPullingTime="8.779465521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:07.175395477 +0000 UTC m=+78.071204940" watchObservedRunningTime="2026-07-27 21:35:07.177271428 +0000 UTC m=+78.073080881"
Jul 27 21:35:07.235626 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:07.235078    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dr8rd" podStartSLOduration=48.376848866 podStartE2EDuration="57.235058202s" totalImagesPullingTime="8.858209336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:07.21969209 +0000 UTC m=+78.115501542" watchObservedRunningTime="2026-07-27 21:35:07.235058202 +0000 UTC m=+78.130867649"
Jul 27 21:35:13.138991 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:13.138892    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh" event={"ID":"e1f716b5-56a7-45a9-941a-e6ac4e380bc8","Type":"ContainerStarted","Data":"c1fd96cccb2856cb6ddcc7bebd649ac111ee07f08869b14aa18d06e9a01a9633"}
Jul 27 21:35:13.139418 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:13.139116    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:13.141252 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:13.141227    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh"
Jul 27 21:35:13.141899 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:13.141875    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5djc" event={"ID":"eabb1707-799d-4ec3-a787-87acc0a7955b","Type":"ContainerStarted","Data":"93033a8ebbf38633559ff70eb8a32143a2f40ddefefc8a7f7a3972af8d085bf0"}
Jul 27 21:35:13.157599 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:13.157552    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d6df559b5-x4vrh" podStartSLOduration=3.660268917 podStartE2EDuration="10.157535777s" totalImagesPullingTime="6.49726686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:13.15568362 +0000 UTC m=+84.051493074" watchObservedRunningTime="2026-07-27 21:35:13.157535777 +0000 UTC m=+84.053345231"
Jul 27 21:35:13.173383 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:13.173336    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r5djc" podStartSLOduration=17.604005684 podStartE2EDuration="38.173323384s" totalImagesPullingTime="20.5693177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:34:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:13.172721761 +0000 UTC m=+84.068531211" watchObservedRunningTime="2026-07-27 21:35:13.173323384 +0000 UTC m=+84.069132835"
Jul 27 21:35:16.592822 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:16.592770    2578 patch_prober.go:28] interesting pod/image-registry-59cd4b4456-zprqx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 21:35:16.593311 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:16.592834    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx" podUID="d7620cb9-8c18-4e77-a04d-df44b78842de" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 21:35:18.120972 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:18.120941    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qdxcv"
Jul 27 21:35:20.033829 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:20.033784    2578 patch_prober.go:28] interesting pod/image-registry-59cd4b4456-zprqx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 21:35:20.034268 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:20.033851    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx" podUID="d7620cb9-8c18-4e77-a04d-df44b78842de" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 21:35:21.379359 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.379322    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cmwkj"]
Jul 27 21:35:21.528910 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.528876    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.532000 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.531972    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 21:35:21.532158 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.532112    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 21:35:21.533243 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.533182    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ghxlq\""
Jul 27 21:35:21.533243 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.533229    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 21:35:21.533387 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.533273    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 21:35:21.607540 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.607495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-textfile\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.607540 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.607540    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-tls\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.607773 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.607574    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a547c558-4bb1-4759-a3c9-611871d54566-sys\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.607773 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.607600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.607773 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.607667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a547c558-4bb1-4759-a3c9-611871d54566-root\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.607773 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.607707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2sfk\" (UniqueName: \"kubernetes.io/projected/a547c558-4bb1-4759-a3c9-611871d54566-kube-api-access-b2sfk\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.607933 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.607776    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-wtmp\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.607933 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.607811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-accelerators-collector-config\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.607933 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.607846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a547c558-4bb1-4759-a3c9-611871d54566-metrics-client-ca\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709014 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.708970    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-textfile\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709019    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-tls\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a547c558-4bb1-4759-a3c9-611871d54566-sys\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709079    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a547c558-4bb1-4759-a3c9-611871d54566-root\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709130    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2sfk\" (UniqueName: \"kubernetes.io/projected/a547c558-4bb1-4759-a3c9-611871d54566-kube-api-access-b2sfk\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-wtmp\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-accelerators-collector-config\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709574 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a547c558-4bb1-4759-a3c9-611871d54566-metrics-client-ca\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709574 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709384    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-textfile\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709676 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709652    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a547c558-4bb1-4759-a3c9-611871d54566-sys\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709775 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709706    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a547c558-4bb1-4759-a3c9-611871d54566-root\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.709968 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.709724    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-wtmp\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.710106 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.710087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a547c558-4bb1-4759-a3c9-611871d54566-metrics-client-ca\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.710157 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.710127    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-accelerators-collector-config\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.712381 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.712358    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-tls\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.712505 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.712439    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a547c558-4bb1-4759-a3c9-611871d54566-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.725724 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.725654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2sfk\" (UniqueName: \"kubernetes.io/projected/a547c558-4bb1-4759-a3c9-611871d54566-kube-api-access-b2sfk\") pod \"node-exporter-cmwkj\" (UID: \"a547c558-4bb1-4759-a3c9-611871d54566\") " pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:21.839812 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:21.839771    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cmwkj"
Jul 27 21:35:25.219568 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:35:25.219536    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda547c558_4bb1_4759_a3c9_611871d54566.slice/crio-a108a82f430e1e4bf118b07d4b3860949dcfdf6d4ee125da539f90952eb65fbd WatchSource:0}: Error finding container a108a82f430e1e4bf118b07d4b3860949dcfdf6d4ee125da539f90952eb65fbd: Status 404 returned error can't find the container with id a108a82f430e1e4bf118b07d4b3860949dcfdf6d4ee125da539f90952eb65fbd
Jul 27 21:35:26.203465 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:26.202806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-6b4dx" event={"ID":"503f8bfb-1452-4193-96c3-5d6dc36bc410","Type":"ContainerStarted","Data":"b414762cbcf0727101755c588e2084c4668fa4464e6e8ef62fa4930f779d4893"}
Jul 27 21:35:26.203465 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:26.203344    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-6b4dx"
Jul 27 21:35:26.205921 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:26.205887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cmwkj" event={"ID":"a547c558-4bb1-4759-a3c9-611871d54566","Type":"ContainerStarted","Data":"a108a82f430e1e4bf118b07d4b3860949dcfdf6d4ee125da539f90952eb65fbd"}
Jul 27 21:35:26.216495 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:26.216468    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-6b4dx"
Jul 27 21:35:26.231102 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:26.230987    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-6b4dx" podStartSLOduration=3.836373934 podStartE2EDuration="23.230965267s" totalImagesPullingTime="19.394591333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:26.229823513 +0000 UTC m=+97.125632976" watchObservedRunningTime="2026-07-27 21:35:26.230965267 +0000 UTC m=+97.126774718"
Jul 27 21:35:26.591890 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:26.591810    2578 patch_prober.go:28] interesting pod/image-registry-59cd4b4456-zprqx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 21:35:26.591890 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:26.591871    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx" podUID="d7620cb9-8c18-4e77-a04d-df44b78842de" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 21:35:27.212328 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:27.212289    2578 generic.go:356] "Generic (PLEG): container finished" podID="a547c558-4bb1-4759-a3c9-611871d54566" containerID="ac38a4ab7002ac59bbf56f052b3a42636061a37fb70a20f18b2ca55405b32e27" exitCode=0
Jul 27 21:35:27.212515 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:27.212381    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cmwkj" event={"ID":"a547c558-4bb1-4759-a3c9-611871d54566","Type":"ContainerDied","Data":"ac38a4ab7002ac59bbf56f052b3a42636061a37fb70a20f18b2ca55405b32e27"}
Jul 27 21:35:28.219963 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:28.219915    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cmwkj" event={"ID":"a547c558-4bb1-4759-a3c9-611871d54566","Type":"ContainerStarted","Data":"ddc67b268f8bf5b5d020c3b68d630fd491c96174ea7270ee4ab16d9f38fa48bf"}
Jul 27 21:35:28.220443 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:28.220022    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cmwkj" event={"ID":"a547c558-4bb1-4759-a3c9-611871d54566","Type":"ContainerStarted","Data":"20ba8b3336bc89adab5a44992f4c50b680d77e9836910fb9c48ad12e44b5ea86"}
Jul 27 21:35:28.245901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:28.245842    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cmwkj" podStartSLOduration=5.75249684 podStartE2EDuration="7.245822127s" totalImagesPullingTime="1.396969366s" totalInitContainerRuntime="96.355921ms" isStatefulPod=true podCreationTimestamp="2026-07-27 21:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:28.243897157 +0000 UTC m=+99.139706633" watchObservedRunningTime="2026-07-27 21:35:28.245822127 +0000 UTC m=+99.141631579"
Jul 27 21:35:29.029413 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:29.029379    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4g95m"
Jul 27 21:35:30.033201 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:30.033168    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:35:30.647381 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:30.647287    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59cd4b4456-zprqx"]
Jul 27 21:35:31.285461 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.285429    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76944b8965-bj9ss"]
Jul 27 21:35:31.337378 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.337340    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76944b8965-bj9ss"]
Jul 27 21:35:31.337560 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.337439    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.340198 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.340168    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 27 21:35:31.340424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.340402    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 27 21:35:31.341441 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.341419    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4dc5n\""
Jul 27 21:35:31.341650 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.341479    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 27 21:35:31.341775 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.341479    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 27 21:35:31.341842 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.341521    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 27 21:35:31.346561 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.346531    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 27 21:35:31.392253 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.392229    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-config\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.392410 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.392267    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzcpk\" (UniqueName: \"kubernetes.io/projected/f41f9cd6-9f4a-4ac8-b39a-530463767854-kube-api-access-mzcpk\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.392410 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.392299    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-trusted-ca-bundle\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.392410 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.392399    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-oauth-config\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.392549 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.392441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-service-ca\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.392549 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.392491    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-oauth-serving-cert\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.392648 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.392569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-serving-cert\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.493269 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.493246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-config\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.493424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.493282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzcpk\" (UniqueName: \"kubernetes.io/projected/f41f9cd6-9f4a-4ac8-b39a-530463767854-kube-api-access-mzcpk\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.493424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.493302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-trusted-ca-bundle\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.493424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.493350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-oauth-config\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.493424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.493384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-service-ca\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.493424 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.493422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-oauth-serving-cert\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.493700 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.493446    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-serving-cert\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.494119 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.494053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-config\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.494334 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.494311    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-service-ca\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.494476 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.494394    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-trusted-ca-bundle\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.494627 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.494601    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-oauth-serving-cert\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.496415 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.496389    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-serving-cert\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.496509 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.496482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-oauth-config\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.502007 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.501982    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzcpk\" (UniqueName: \"kubernetes.io/projected/f41f9cd6-9f4a-4ac8-b39a-530463767854-kube-api-access-mzcpk\") pod \"console-76944b8965-bj9ss\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") " pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.648815 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.648748    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:31.817076 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:31.817039    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76944b8965-bj9ss"]
Jul 27 21:35:31.823523 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:35:31.823492    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf41f9cd6_9f4a_4ac8_b39a_530463767854.slice/crio-577064d0760d3482a84c13d7c1be1824058cf084d97fe7a29204fdf7a9ab9d95 WatchSource:0}: Error finding container 577064d0760d3482a84c13d7c1be1824058cf084d97fe7a29204fdf7a9ab9d95: Status 404 returned error can't find the container with id 577064d0760d3482a84c13d7c1be1824058cf084d97fe7a29204fdf7a9ab9d95
Jul 27 21:35:32.239677 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:32.239645    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76944b8965-bj9ss" event={"ID":"f41f9cd6-9f4a-4ac8-b39a-530463767854","Type":"ContainerStarted","Data":"577064d0760d3482a84c13d7c1be1824058cf084d97fe7a29204fdf7a9ab9d95"}
Jul 27 21:35:36.261884 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:36.261843    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76944b8965-bj9ss" event={"ID":"f41f9cd6-9f4a-4ac8-b39a-530463767854","Type":"ContainerStarted","Data":"764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf"}
Jul 27 21:35:36.286060 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:36.286002    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76944b8965-bj9ss" podStartSLOduration=1.346649876 podStartE2EDuration="5.285983268s" totalImagesPullingTime="3.939333392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:35:36.285438211 +0000 UTC m=+107.181247662" watchObservedRunningTime="2026-07-27 21:35:36.285983268 +0000 UTC m=+107.181792719"
Jul 27 21:35:41.649335 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:41.649292    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:41.649893 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:41.649357    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:41.654381 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:41.654355    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:42.288388 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:42.288362    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:35:45.296833 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:45.296802    2578 generic.go:356] "Generic (PLEG): container finished" podID="8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29" containerID="6d8bc6bb576b3a21cc5878a44ee1c46825ba6bb92da065c03fc5d69ce77486d3" exitCode=0
Jul 27 21:35:45.297145 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:45.296872    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6" event={"ID":"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29","Type":"ContainerDied","Data":"6d8bc6bb576b3a21cc5878a44ee1c46825ba6bb92da065c03fc5d69ce77486d3"}
Jul 27 21:35:45.297187 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:45.297178    2578 scope.go:117] "RemoveContainer" containerID="6d8bc6bb576b3a21cc5878a44ee1c46825ba6bb92da065c03fc5d69ce77486d3"
Jul 27 21:35:46.303901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:46.303353    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zmgn6" event={"ID":"8d8d6e52-9b50-4c2e-a948-86a1bd2c7f29","Type":"ContainerStarted","Data":"f938189954cee9f0a6b1012c5e263409656df34a8fc382e5bc20fa7f9aa4e7f4"}
Jul 27 21:35:55.670542 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:55.670481    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx" podUID="d7620cb9-8c18-4e77-a04d-df44b78842de" containerName="registry" containerID="cri-o://9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e" gracePeriod=30
Jul 27 21:35:55.944260 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:55.944208    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:35:56.076807 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.076779    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-installation-pull-secrets\") pod \"d7620cb9-8c18-4e77-a04d-df44b78842de\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") "
Jul 27 21:35:56.076956 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.076814    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzfc4\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-kube-api-access-kzfc4\") pod \"d7620cb9-8c18-4e77-a04d-df44b78842de\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") "
Jul 27 21:35:56.076956 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.076836    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-image-registry-private-configuration\") pod \"d7620cb9-8c18-4e77-a04d-df44b78842de\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") "
Jul 27 21:35:56.076956 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.076860    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7620cb9-8c18-4e77-a04d-df44b78842de-ca-trust-extracted\") pod \"d7620cb9-8c18-4e77-a04d-df44b78842de\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") "
Jul 27 21:35:56.076956 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.076886    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-certificates\") pod \"d7620cb9-8c18-4e77-a04d-df44b78842de\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") "
Jul 27 21:35:56.076956 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.076926    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-bound-sa-token\") pod \"d7620cb9-8c18-4e77-a04d-df44b78842de\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") "
Jul 27 21:35:56.076956 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.076955    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") pod \"d7620cb9-8c18-4e77-a04d-df44b78842de\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") "
Jul 27 21:35:56.077262 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.077020    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-trusted-ca\") pod \"d7620cb9-8c18-4e77-a04d-df44b78842de\" (UID: \"d7620cb9-8c18-4e77-a04d-df44b78842de\") "
Jul 27 21:35:56.077832 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.077762    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d7620cb9-8c18-4e77-a04d-df44b78842de" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:35:56.077832 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.077768    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d7620cb9-8c18-4e77-a04d-df44b78842de" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:35:56.079793 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.079727    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d7620cb9-8c18-4e77-a04d-df44b78842de" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:35:56.079913 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.079784    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-kube-api-access-kzfc4" (OuterVolumeSpecName: "kube-api-access-kzfc4") pod "d7620cb9-8c18-4e77-a04d-df44b78842de" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de"). InnerVolumeSpecName "kube-api-access-kzfc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:35:56.080004 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.079981    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d7620cb9-8c18-4e77-a04d-df44b78842de" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:35:56.080072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.080052    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d7620cb9-8c18-4e77-a04d-df44b78842de" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:35:56.080447 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.080422    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d7620cb9-8c18-4e77-a04d-df44b78842de" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:35:56.086125 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.086101    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7620cb9-8c18-4e77-a04d-df44b78842de-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d7620cb9-8c18-4e77-a04d-df44b78842de" (UID: "d7620cb9-8c18-4e77-a04d-df44b78842de"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:35:56.178000 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.177976    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-trusted-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:35:56.178100 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.178005    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-installation-pull-secrets\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:35:56.178100 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.178016    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzfc4\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-kube-api-access-kzfc4\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:35:56.178100 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.178032    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7620cb9-8c18-4e77-a04d-df44b78842de-image-registry-private-configuration\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:35:56.178100 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.178045    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7620cb9-8c18-4e77-a04d-df44b78842de-ca-trust-extracted\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:35:56.178100 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.178053    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-certificates\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:35:56.178100 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.178066    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-bound-sa-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:35:56.178100 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.178078    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7620cb9-8c18-4e77-a04d-df44b78842de-registry-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:35:56.341062 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.341003    2578 generic.go:356] "Generic (PLEG): container finished" podID="d7620cb9-8c18-4e77-a04d-df44b78842de" containerID="9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e" exitCode=0
Jul 27 21:35:56.341062 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.341047    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx"
Jul 27 21:35:56.341182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.341065    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx" event={"ID":"d7620cb9-8c18-4e77-a04d-df44b78842de","Type":"ContainerDied","Data":"9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e"}
Jul 27 21:35:56.341182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.341113    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59cd4b4456-zprqx" event={"ID":"d7620cb9-8c18-4e77-a04d-df44b78842de","Type":"ContainerDied","Data":"6eb88b43a74502fdddad9daeba79724167c04e6866c4943430370fcafdc8c785"}
Jul 27 21:35:56.341182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.341139    2578 scope.go:117] "RemoveContainer" containerID="9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e"
Jul 27 21:35:56.349218 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.349203    2578 scope.go:117] "RemoveContainer" containerID="9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e"
Jul 27 21:35:56.349506 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:35:56.349484    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e\": container with ID starting with 9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e not found: ID does not exist" containerID="9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e"
Jul 27 21:35:56.349570 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.349517    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e"} err="failed to get container status \"9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e\": rpc error: code = NotFound desc = could not find container \"9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e\": container with ID starting with 9c825575a46ea32c8015d1392f53e3c28a6b25f8eb777528a5422a74820b3d6e not found: ID does not exist"
Jul 27 21:35:56.368485 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.368455    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59cd4b4456-zprqx"]
Jul 27 21:35:56.377304 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:56.377285    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59cd4b4456-zprqx"]
Jul 27 21:35:57.609015 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:35:57.608980    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7620cb9-8c18-4e77-a04d-df44b78842de" path="/var/lib/kubelet/pods/d7620cb9-8c18-4e77-a04d-df44b78842de/volumes"
Jul 27 21:36:01.361295 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:01.361264    2578 generic.go:356] "Generic (PLEG): container finished" podID="2e1b4aee-0b22-46b0-80f7-362fd14d99fc" containerID="3f2817cc9329246a38be13175f2b51d73a18f7c8b157ab93f6b6c041b12e0e94" exitCode=0
Jul 27 21:36:01.361683 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:01.361312    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c" event={"ID":"2e1b4aee-0b22-46b0-80f7-362fd14d99fc","Type":"ContainerDied","Data":"3f2817cc9329246a38be13175f2b51d73a18f7c8b157ab93f6b6c041b12e0e94"}
Jul 27 21:36:01.361683 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:01.361563    2578 scope.go:117] "RemoveContainer" containerID="3f2817cc9329246a38be13175f2b51d73a18f7c8b157ab93f6b6c041b12e0e94"
Jul 27 21:36:02.366782 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:02.366751    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c" event={"ID":"2e1b4aee-0b22-46b0-80f7-362fd14d99fc","Type":"ContainerStarted","Data":"1fde451ac2f360582fd70ed39c7b2383c137f426e57a67f4349cdc69aae8c7a9"}
Jul 27 21:36:05.379417 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:05.379384    2578 generic.go:356] "Generic (PLEG): container finished" podID="3a511784-485f-4334-b266-e9e7dea58199" containerID="4bf1b2c25317c2d711cfa32d48164702ffde50127b4b9838b6e10dec451ec0d0" exitCode=0
Jul 27 21:36:05.379794 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:05.379456    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-cpgfx" event={"ID":"3a511784-485f-4334-b266-e9e7dea58199","Type":"ContainerDied","Data":"4bf1b2c25317c2d711cfa32d48164702ffde50127b4b9838b6e10dec451ec0d0"}
Jul 27 21:36:05.379794 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:05.379771    2578 scope.go:117] "RemoveContainer" containerID="4bf1b2c25317c2d711cfa32d48164702ffde50127b4b9838b6e10dec451ec0d0"
Jul 27 21:36:06.384626 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:06.384592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-cpgfx" event={"ID":"3a511784-485f-4334-b266-e9e7dea58199","Type":"ContainerStarted","Data":"8ad5ae4213ab48d457ae2d929d2609c3bbf9df18bca27a5c140ea051aa7c7134"}
Jul 27 21:36:41.705141 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.705060    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79976778f9-9tsft"]
Jul 27 21:36:41.705596 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.705353    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7620cb9-8c18-4e77-a04d-df44b78842de" containerName="registry"
Jul 27 21:36:41.705596 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.705364    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7620cb9-8c18-4e77-a04d-df44b78842de" containerName="registry"
Jul 27 21:36:41.705596 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.705458    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7620cb9-8c18-4e77-a04d-df44b78842de" containerName="registry"
Jul 27 21:36:41.707509 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.707484    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.723009 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.722984    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79976778f9-9tsft"]
Jul 27 21:36:41.800833 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.800809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-serving-cert\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.800965 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.800835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f4q8\" (UniqueName: \"kubernetes.io/projected/0ec029cc-6372-4ccb-98b6-258c223baefa-kube-api-access-4f4q8\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.800965 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.800860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-oauth-serving-cert\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.800965 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.800880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-trusted-ca-bundle\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.801097 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.800974    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-service-ca\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.801097 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.801007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-console-config\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.801097 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.801026    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-oauth-config\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.902322 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.902301    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-oauth-config\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.902445 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.902339    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-serving-cert\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.902445 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.902357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f4q8\" (UniqueName: \"kubernetes.io/projected/0ec029cc-6372-4ccb-98b6-258c223baefa-kube-api-access-4f4q8\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.902445 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.902379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-oauth-serving-cert\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.902445 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.902406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-trusted-ca-bundle\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.902445 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.902443    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-service-ca\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.902693 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.902475    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-console-config\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.903138 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.903114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-oauth-serving-cert\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.903249 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.903201    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-service-ca\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.903299 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.903278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-console-config\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.903776 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.903754    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-trusted-ca-bundle\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.905158 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.905132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-oauth-config\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.905317 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.905295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-serving-cert\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:41.914247 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:41.914229    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f4q8\" (UniqueName: \"kubernetes.io/projected/0ec029cc-6372-4ccb-98b6-258c223baefa-kube-api-access-4f4q8\") pod \"console-79976778f9-9tsft\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") " pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:42.018416 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:42.018366    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:42.163203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:42.163170    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79976778f9-9tsft"]
Jul 27 21:36:42.166153 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:36:42.166126    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ec029cc_6372_4ccb_98b6_258c223baefa.slice/crio-0673a0eb490f174f8c74154a436ee84e0f5324314373601f45c2121e19660899 WatchSource:0}: Error finding container 0673a0eb490f174f8c74154a436ee84e0f5324314373601f45c2121e19660899: Status 404 returned error can't find the container with id 0673a0eb490f174f8c74154a436ee84e0f5324314373601f45c2121e19660899
Jul 27 21:36:42.526716 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:42.526682    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79976778f9-9tsft" event={"ID":"0ec029cc-6372-4ccb-98b6-258c223baefa","Type":"ContainerStarted","Data":"c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0"}
Jul 27 21:36:42.526891 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:42.526766    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79976778f9-9tsft" event={"ID":"0ec029cc-6372-4ccb-98b6-258c223baefa","Type":"ContainerStarted","Data":"0673a0eb490f174f8c74154a436ee84e0f5324314373601f45c2121e19660899"}
Jul 27 21:36:42.554159 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:42.554112    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79976778f9-9tsft" podStartSLOduration=1.554095852 podStartE2EDuration="1.554095852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:36:42.553087613 +0000 UTC m=+173.448897073" watchObservedRunningTime="2026-07-27 21:36:42.554095852 +0000 UTC m=+173.449905303"
Jul 27 21:36:52.019101 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:52.019064    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:52.019604 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:52.019129    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:52.023849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:52.023823    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:52.570045 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:52.570018    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:36:52.620397 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:36:52.620367    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76944b8965-bj9ss"]
Jul 27 21:37:17.643306 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.643263    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76944b8965-bj9ss" podUID="f41f9cd6-9f4a-4ac8-b39a-530463767854" containerName="console" containerID="cri-o://764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf" gracePeriod=15
Jul 27 21:37:17.888909 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.888888    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:37:17.960153 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960130    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-oauth-serving-cert\") pod \"f41f9cd6-9f4a-4ac8-b39a-530463767854\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") "
Jul 27 21:37:17.960284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960172    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-config\") pod \"f41f9cd6-9f4a-4ac8-b39a-530463767854\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") "
Jul 27 21:37:17.960284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960202    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-oauth-config\") pod \"f41f9cd6-9f4a-4ac8-b39a-530463767854\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") "
Jul 27 21:37:17.960284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960235    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-serving-cert\") pod \"f41f9cd6-9f4a-4ac8-b39a-530463767854\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") "
Jul 27 21:37:17.960284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960265    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-trusted-ca-bundle\") pod \"f41f9cd6-9f4a-4ac8-b39a-530463767854\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") "
Jul 27 21:37:17.960284 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960280    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-service-ca\") pod \"f41f9cd6-9f4a-4ac8-b39a-530463767854\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") "
Jul 27 21:37:17.960519 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960310    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzcpk\" (UniqueName: \"kubernetes.io/projected/f41f9cd6-9f4a-4ac8-b39a-530463767854-kube-api-access-mzcpk\") pod \"f41f9cd6-9f4a-4ac8-b39a-530463767854\" (UID: \"f41f9cd6-9f4a-4ac8-b39a-530463767854\") "
Jul 27 21:37:17.960576 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960524    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-config" (OuterVolumeSpecName: "console-config") pod "f41f9cd6-9f4a-4ac8-b39a-530463767854" (UID: "f41f9cd6-9f4a-4ac8-b39a-530463767854"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:37:17.960714 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960679    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f41f9cd6-9f4a-4ac8-b39a-530463767854" (UID: "f41f9cd6-9f4a-4ac8-b39a-530463767854"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:37:17.960843 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960769    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f41f9cd6-9f4a-4ac8-b39a-530463767854" (UID: "f41f9cd6-9f4a-4ac8-b39a-530463767854"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:37:17.960843 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.960794    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-service-ca" (OuterVolumeSpecName: "service-ca") pod "f41f9cd6-9f4a-4ac8-b39a-530463767854" (UID: "f41f9cd6-9f4a-4ac8-b39a-530463767854"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:37:17.962607 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.962585    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f41f9cd6-9f4a-4ac8-b39a-530463767854" (UID: "f41f9cd6-9f4a-4ac8-b39a-530463767854"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:37:17.962686 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.962647    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f41f9cd6-9f4a-4ac8-b39a-530463767854-kube-api-access-mzcpk" (OuterVolumeSpecName: "kube-api-access-mzcpk") pod "f41f9cd6-9f4a-4ac8-b39a-530463767854" (UID: "f41f9cd6-9f4a-4ac8-b39a-530463767854"). InnerVolumeSpecName "kube-api-access-mzcpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:37:17.962767 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:17.962721    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f41f9cd6-9f4a-4ac8-b39a-530463767854" (UID: "f41f9cd6-9f4a-4ac8-b39a-530463767854"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:37:18.061374 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.061351    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-serving-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:37:18.061374 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.061369    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-trusted-ca-bundle\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:37:18.061503 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.061379    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:37:18.061503 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.061388    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzcpk\" (UniqueName: \"kubernetes.io/projected/f41f9cd6-9f4a-4ac8-b39a-530463767854-kube-api-access-mzcpk\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:37:18.061503 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.061396    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-oauth-serving-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:37:18.061503 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.061404    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-config\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:37:18.061503 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.061412    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f41f9cd6-9f4a-4ac8-b39a-530463767854-console-oauth-config\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:37:18.679953 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.679921    2578 generic.go:356] "Generic (PLEG): container finished" podID="f41f9cd6-9f4a-4ac8-b39a-530463767854" containerID="764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf" exitCode=0
Jul 27 21:37:18.680351 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.679969    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76944b8965-bj9ss"
Jul 27 21:37:18.680351 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.679983    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76944b8965-bj9ss" event={"ID":"f41f9cd6-9f4a-4ac8-b39a-530463767854","Type":"ContainerDied","Data":"764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf"}
Jul 27 21:37:18.680351 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.680022    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76944b8965-bj9ss" event={"ID":"f41f9cd6-9f4a-4ac8-b39a-530463767854","Type":"ContainerDied","Data":"577064d0760d3482a84c13d7c1be1824058cf084d97fe7a29204fdf7a9ab9d95"}
Jul 27 21:37:18.680351 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.680042    2578 scope.go:117] "RemoveContainer" containerID="764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf"
Jul 27 21:37:18.689526 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.689509    2578 scope.go:117] "RemoveContainer" containerID="764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf"
Jul 27 21:37:18.689823 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:37:18.689800    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf\": container with ID starting with 764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf not found: ID does not exist" containerID="764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf"
Jul 27 21:37:18.689896 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.689835    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf"} err="failed to get container status \"764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf\": rpc error: code = NotFound desc = could not find container \"764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf\": container with ID starting with 764fd6e4c1ffc3d693eb151f685f31896321afceb8c0d1fadcdb3b4eeb471bdf not found: ID does not exist"
Jul 27 21:37:18.703106 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.703057    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76944b8965-bj9ss"]
Jul 27 21:37:18.706101 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:18.705594    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76944b8965-bj9ss"]
Jul 27 21:37:19.606306 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:37:19.606258    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f41f9cd6-9f4a-4ac8-b39a-530463767854" path="/var/lib/kubelet/pods/f41f9cd6-9f4a-4ac8-b39a-530463767854/volumes"
Jul 27 21:38:21.653754 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.653664    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"]
Jul 27 21:38:21.654215 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.654003    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f41f9cd6-9f4a-4ac8-b39a-530463767854" containerName="console"
Jul 27 21:38:21.654215 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.654016    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f41f9cd6-9f4a-4ac8-b39a-530463767854" containerName="console"
Jul 27 21:38:21.654215 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.654077    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f41f9cd6-9f4a-4ac8-b39a-530463767854" containerName="console"
Jul 27 21:38:21.655940 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.655919    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.659404 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.659384    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 27 21:38:21.660546 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.660528    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bdqdb\""
Jul 27 21:38:21.660619 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.660561    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 27 21:38:21.670921 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.670902    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"]
Jul 27 21:38:21.701649 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.701623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.701765 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.701663    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.701765 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.701687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwwwk\" (UniqueName: \"kubernetes.io/projected/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-kube-api-access-pwwwk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.802390 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.802364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.802472 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.802393    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwwwk\" (UniqueName: \"kubernetes.io/projected/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-kube-api-access-pwwwk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.802472 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.802437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.802715 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.802696    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.802781 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.802757    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.824160 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.824126    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwwwk\" (UniqueName: \"kubernetes.io/projected/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-kube-api-access-pwwwk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:21.964698 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:21.964641    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:22.093393 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:22.093364    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"]
Jul 27 21:38:22.095399 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:38:22.095375    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b0551b_c0c2_4ab6_8fbe_a9aa31800806.slice/crio-a1e0931aa165377ca1301c810449ddd6e16bc661c0aedceca1182117b35d8e37 WatchSource:0}: Error finding container a1e0931aa165377ca1301c810449ddd6e16bc661c0aedceca1182117b35d8e37: Status 404 returned error can't find the container with id a1e0931aa165377ca1301c810449ddd6e16bc661c0aedceca1182117b35d8e37
Jul 27 21:38:22.920099 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:22.920059    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7" event={"ID":"96b0551b-c0c2-4ab6-8fbe-a9aa31800806","Type":"ContainerStarted","Data":"a1e0931aa165377ca1301c810449ddd6e16bc661c0aedceca1182117b35d8e37"}
Jul 27 21:38:28.946901 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:28.946862    2578 generic.go:356] "Generic (PLEG): container finished" podID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerID="424b4d8d4b83cde79d9f5fc72e461dd06a5dbf47ae4407672487eb889b917615" exitCode=0
Jul 27 21:38:28.947316 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:28.946921    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7" event={"ID":"96b0551b-c0c2-4ab6-8fbe-a9aa31800806","Type":"ContainerDied","Data":"424b4d8d4b83cde79d9f5fc72e461dd06a5dbf47ae4407672487eb889b917615"}
Jul 27 21:38:30.957078 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:30.957049    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7" event={"ID":"96b0551b-c0c2-4ab6-8fbe-a9aa31800806","Type":"ContainerStarted","Data":"598eff0e98e02afd2e9fb06f890de86fc5a86e1d72a70ca7272d5cc371110e53"}
Jul 27 21:38:31.963259 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:31.963229    2578 generic.go:356] "Generic (PLEG): container finished" podID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerID="598eff0e98e02afd2e9fb06f890de86fc5a86e1d72a70ca7272d5cc371110e53" exitCode=0
Jul 27 21:38:31.963614 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:31.963263    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7" event={"ID":"96b0551b-c0c2-4ab6-8fbe-a9aa31800806","Type":"ContainerDied","Data":"598eff0e98e02afd2e9fb06f890de86fc5a86e1d72a70ca7272d5cc371110e53"}
Jul 27 21:38:32.150103 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.150071    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b8f8c"]
Jul 27 21:38:32.152503 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.152484    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b8f8c"
Jul 27 21:38:32.155249 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.155152    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 21:38:32.155249 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.155169    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 21:38:32.156203 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.156183    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-667rr\""
Jul 27 21:38:32.163616 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.163571    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b8f8c"]
Jul 27 21:38:32.178894 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.178870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztcff\" (UniqueName: \"kubernetes.io/projected/b9210b5c-24c7-48ec-a239-3e6406b2a0b3-kube-api-access-ztcff\") pod \"cert-manager-55b4d954d5-b8f8c\" (UID: \"b9210b5c-24c7-48ec-a239-3e6406b2a0b3\") " pod="cert-manager/cert-manager-55b4d954d5-b8f8c"
Jul 27 21:38:32.179053 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.178934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9210b5c-24c7-48ec-a239-3e6406b2a0b3-bound-sa-token\") pod \"cert-manager-55b4d954d5-b8f8c\" (UID: \"b9210b5c-24c7-48ec-a239-3e6406b2a0b3\") " pod="cert-manager/cert-manager-55b4d954d5-b8f8c"
Jul 27 21:38:32.280290 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.280232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9210b5c-24c7-48ec-a239-3e6406b2a0b3-bound-sa-token\") pod \"cert-manager-55b4d954d5-b8f8c\" (UID: \"b9210b5c-24c7-48ec-a239-3e6406b2a0b3\") " pod="cert-manager/cert-manager-55b4d954d5-b8f8c"
Jul 27 21:38:32.280408 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.280294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztcff\" (UniqueName: \"kubernetes.io/projected/b9210b5c-24c7-48ec-a239-3e6406b2a0b3-kube-api-access-ztcff\") pod \"cert-manager-55b4d954d5-b8f8c\" (UID: \"b9210b5c-24c7-48ec-a239-3e6406b2a0b3\") " pod="cert-manager/cert-manager-55b4d954d5-b8f8c"
Jul 27 21:38:32.289487 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.289463    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9210b5c-24c7-48ec-a239-3e6406b2a0b3-bound-sa-token\") pod \"cert-manager-55b4d954d5-b8f8c\" (UID: \"b9210b5c-24c7-48ec-a239-3e6406b2a0b3\") " pod="cert-manager/cert-manager-55b4d954d5-b8f8c"
Jul 27 21:38:32.289603 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.289558    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztcff\" (UniqueName: \"kubernetes.io/projected/b9210b5c-24c7-48ec-a239-3e6406b2a0b3-kube-api-access-ztcff\") pod \"cert-manager-55b4d954d5-b8f8c\" (UID: \"b9210b5c-24c7-48ec-a239-3e6406b2a0b3\") " pod="cert-manager/cert-manager-55b4d954d5-b8f8c"
Jul 27 21:38:32.470140 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.470106    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b8f8c"
Jul 27 21:38:32.604508 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.604469    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b8f8c"]
Jul 27 21:38:32.608622 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:38:32.608596    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9210b5c_24c7_48ec_a239_3e6406b2a0b3.slice/crio-f270f32d29288be95f5bdb773dccf91bf10d5cf3559e1402b43dc6ef3f780685 WatchSource:0}: Error finding container f270f32d29288be95f5bdb773dccf91bf10d5cf3559e1402b43dc6ef3f780685: Status 404 returned error can't find the container with id f270f32d29288be95f5bdb773dccf91bf10d5cf3559e1402b43dc6ef3f780685
Jul 27 21:38:32.968666 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:32.968626    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b8f8c" event={"ID":"b9210b5c-24c7-48ec-a239-3e6406b2a0b3","Type":"ContainerStarted","Data":"f270f32d29288be95f5bdb773dccf91bf10d5cf3559e1402b43dc6ef3f780685"}
Jul 27 21:38:38.999049 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:38.999007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b8f8c" event={"ID":"b9210b5c-24c7-48ec-a239-3e6406b2a0b3","Type":"ContainerStarted","Data":"8f35593752ad76f87a6eff5e8964e2973deea060a6ff5c5a47ec26829538db67"}
Jul 27 21:38:39.001414 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:39.001385    2578 generic.go:356] "Generic (PLEG): container finished" podID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerID="783105227ac836d909cd5772bb51043118808486f3be013e7c1409ff685357db" exitCode=0
Jul 27 21:38:39.001554 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:39.001466    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7" event={"ID":"96b0551b-c0c2-4ab6-8fbe-a9aa31800806","Type":"ContainerDied","Data":"783105227ac836d909cd5772bb51043118808486f3be013e7c1409ff685357db"}
Jul 27 21:38:39.017233 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:39.017188    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-b8f8c" podStartSLOduration=1.4125189009999999 podStartE2EDuration="7.017170863s" totalImagesPullingTime="5.604651962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:38:39.014847724 +0000 UTC m=+289.910657174" watchObservedRunningTime="2026-07-27 21:38:39.017170863 +0000 UTC m=+289.912980314"
Jul 27 21:38:40.130573 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.130551    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:40.241200 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.241174    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwwwk\" (UniqueName: \"kubernetes.io/projected/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-kube-api-access-pwwwk\") pod \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") "
Jul 27 21:38:40.241333 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.241225    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-util\") pod \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") "
Jul 27 21:38:40.241333 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.241254    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-bundle\") pod \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\" (UID: \"96b0551b-c0c2-4ab6-8fbe-a9aa31800806\") "
Jul 27 21:38:40.241621 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.241601    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-bundle" (OuterVolumeSpecName: "bundle") pod "96b0551b-c0c2-4ab6-8fbe-a9aa31800806" (UID: "96b0551b-c0c2-4ab6-8fbe-a9aa31800806"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:38:40.243419 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.243397    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-kube-api-access-pwwwk" (OuterVolumeSpecName: "kube-api-access-pwwwk") pod "96b0551b-c0c2-4ab6-8fbe-a9aa31800806" (UID: "96b0551b-c0c2-4ab6-8fbe-a9aa31800806"). InnerVolumeSpecName "kube-api-access-pwwwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:38:40.245137 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.245115    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-util" (OuterVolumeSpecName: "util") pod "96b0551b-c0c2-4ab6-8fbe-a9aa31800806" (UID: "96b0551b-c0c2-4ab6-8fbe-a9aa31800806"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:38:40.341910 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.341857    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwwwk\" (UniqueName: \"kubernetes.io/projected/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-kube-api-access-pwwwk\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:38:40.341910 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.341880    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-util\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:38:40.341910 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:40.341890    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96b0551b-c0c2-4ab6-8fbe-a9aa31800806-bundle\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:38:41.011889 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:41.011861    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7"
Jul 27 21:38:41.012028 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:41.011863    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffsdl7" event={"ID":"96b0551b-c0c2-4ab6-8fbe-a9aa31800806","Type":"ContainerDied","Data":"a1e0931aa165377ca1301c810449ddd6e16bc661c0aedceca1182117b35d8e37"}
Jul 27 21:38:41.012028 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:41.011982    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1e0931aa165377ca1301c810449ddd6e16bc661c0aedceca1182117b35d8e37"
Jul 27 21:38:49.441617 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:49.441585    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:38:49.445994 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:49.445970    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:38:49.453409 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:38:49.453393    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 21:39:04.352060 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.352031    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"]
Jul 27 21:39:04.353360 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.352342    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerName="extract"
Jul 27 21:39:04.353360 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.352357    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerName="extract"
Jul 27 21:39:04.353360 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.352372    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerName="util"
Jul 27 21:39:04.353360 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.352377    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerName="util"
Jul 27 21:39:04.353360 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.352385    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerName="pull"
Jul 27 21:39:04.353360 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.352390    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerName="pull"
Jul 27 21:39:04.353360 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.352448    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="96b0551b-c0c2-4ab6-8fbe-a9aa31800806" containerName="extract"
Jul 27 21:39:04.354001 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.353986    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.361777 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.361758    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 21:39:04.361910 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.361892    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 21:39:04.361975 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.361962    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 21:39:04.363179 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.363164    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:39:04.363220 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.363184    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 21:39:04.363220 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.363189    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-txdpd\""
Jul 27 21:39:04.383171 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.383150    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"]
Jul 27 21:39:04.424661 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.424637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh4lk\" (UniqueName: \"kubernetes.io/projected/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-kube-api-access-lh4lk\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.424817 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.424690    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-metrics-cert\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.424817 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.424786    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-cert\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.424817 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.424813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-manager-config\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.525262 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.525226    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-metrics-cert\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.525262 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.525267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-cert\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.525460 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.525298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-manager-config\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.525460 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.525355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh4lk\" (UniqueName: \"kubernetes.io/projected/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-kube-api-access-lh4lk\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.526072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.526050    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-manager-config\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.527924 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.527907    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-metrics-cert\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.528065 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.528044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-cert\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.542374 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.542340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh4lk\" (UniqueName: \"kubernetes.io/projected/4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44-kube-api-access-lh4lk\") pod \"lws-controller-manager-7c57b59594-lvwjn\" (UID: \"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44\") " pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.663387 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.663360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:04.846907 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.846877    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"]
Jul 27 21:39:04.851788 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:39:04.851756    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f8f7fc9_f006_42bf_bbf1_ff6f9ecbec44.slice/crio-a4e71dfb52ce2e18515bb3aad3876e6d94b7a77635ffd873363cd76f5348e832 WatchSource:0}: Error finding container a4e71dfb52ce2e18515bb3aad3876e6d94b7a77635ffd873363cd76f5348e832: Status 404 returned error can't find the container with id a4e71dfb52ce2e18515bb3aad3876e6d94b7a77635ffd873363cd76f5348e832
Jul 27 21:39:04.854271 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:04.854257    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:39:05.105074 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:05.104988    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn" event={"ID":"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44","Type":"ContainerStarted","Data":"a4e71dfb52ce2e18515bb3aad3876e6d94b7a77635ffd873363cd76f5348e832"}
Jul 27 21:39:06.143234 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.143198    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"]
Jul 27 21:39:06.146014 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.145988    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.150350 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.150316    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 21:39:06.150653 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.150632    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xl2cn\""
Jul 27 21:39:06.151335 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.151319    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 21:39:06.151425 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.151381    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 21:39:06.151585 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.151559    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 21:39:06.177505 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.177479    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"]
Jul 27 21:39:06.239860 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.239827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bed8d6e8-1317-4d45-913d-db49e3c2e28f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.240050 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.239891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bed8d6e8-1317-4d45-913d-db49e3c2e28f-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.240050 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.239968    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bed8d6e8-1317-4d45-913d-db49e3c2e28f-tmp\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.240050 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.240004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpcnh\" (UniqueName: \"kubernetes.io/projected/bed8d6e8-1317-4d45-913d-db49e3c2e28f-kube-api-access-bpcnh\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.240229 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.240066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bed8d6e8-1317-4d45-913d-db49e3c2e28f-manifests\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.340659 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.340617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bed8d6e8-1317-4d45-913d-db49e3c2e28f-manifests\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.340876 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.340704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bed8d6e8-1317-4d45-913d-db49e3c2e28f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.340876 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.340772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bed8d6e8-1317-4d45-913d-db49e3c2e28f-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.340876 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.340810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bed8d6e8-1317-4d45-913d-db49e3c2e28f-tmp\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.340876 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.340834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcnh\" (UniqueName: \"kubernetes.io/projected/bed8d6e8-1317-4d45-913d-db49e3c2e28f-kube-api-access-bpcnh\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.341132 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.341096    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bed8d6e8-1317-4d45-913d-db49e3c2e28f-manifests\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.343889 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.343860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bed8d6e8-1317-4d45-913d-db49e3c2e28f-tmp\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.344067 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.344039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bed8d6e8-1317-4d45-913d-db49e3c2e28f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.344245 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.344129    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bed8d6e8-1317-4d45-913d-db49e3c2e28f-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.350064 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.350038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpcnh\" (UniqueName: \"kubernetes.io/projected/bed8d6e8-1317-4d45-913d-db49e3c2e28f-kube-api-access-bpcnh\") pod \"opendatahub-operator-controller-manager-5d57c545b5-7pdmp\" (UID: \"bed8d6e8-1317-4d45-913d-db49e3c2e28f\") " pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.459426 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.459396    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:06.618817 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:06.618776    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"]
Jul 27 21:39:06.624841 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:39:06.624801    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed8d6e8_1317_4d45_913d_db49e3c2e28f.slice/crio-5256eb64ba42958b27c3650dd23bef466c9e9e502ce6cfb491ddbb2b6cc319b9 WatchSource:0}: Error finding container 5256eb64ba42958b27c3650dd23bef466c9e9e502ce6cfb491ddbb2b6cc319b9: Status 404 returned error can't find the container with id 5256eb64ba42958b27c3650dd23bef466c9e9e502ce6cfb491ddbb2b6cc319b9
Jul 27 21:39:07.114638 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:07.114599    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp" event={"ID":"bed8d6e8-1317-4d45-913d-db49e3c2e28f","Type":"ContainerStarted","Data":"5256eb64ba42958b27c3650dd23bef466c9e9e502ce6cfb491ddbb2b6cc319b9"}
Jul 27 21:39:07.116111 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:07.116086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn" event={"ID":"4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44","Type":"ContainerStarted","Data":"dbe7d31973d16e33987f0bf34e7dcdc18c2eec6876e7bc67500376e9ab2cfdce"}
Jul 27 21:39:07.116253 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:07.116218    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:07.133430 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:07.133374    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn" podStartSLOduration=1.583358214 podStartE2EDuration="3.133356988s" totalImagesPullingTime="1.549998774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:39:07.132702111 +0000 UTC m=+318.028511553" watchObservedRunningTime="2026-07-27 21:39:07.133356988 +0000 UTC m=+318.029166438"
Jul 27 21:39:10.132455 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:10.132420    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp" event={"ID":"bed8d6e8-1317-4d45-913d-db49e3c2e28f","Type":"ContainerStarted","Data":"67f355853b893dd8141183be1843f2a5350c5dcbe94045b3d4bb4a5c6a1f9ed7"}
Jul 27 21:39:11.138515 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:11.138422    2578 generic.go:356] "Generic (PLEG): container finished" podID="bed8d6e8-1317-4d45-913d-db49e3c2e28f" containerID="67f355853b893dd8141183be1843f2a5350c5dcbe94045b3d4bb4a5c6a1f9ed7" exitCode=0
Jul 27 21:39:11.138515 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:11.138500    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp" event={"ID":"bed8d6e8-1317-4d45-913d-db49e3c2e28f","Type":"ContainerDied","Data":"67f355853b893dd8141183be1843f2a5350c5dcbe94045b3d4bb4a5c6a1f9ed7"}
Jul 27 21:39:12.145077 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:12.145044    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp" event={"ID":"bed8d6e8-1317-4d45-913d-db49e3c2e28f","Type":"ContainerStarted","Data":"c60f0b43036e453ec28ae8ac4e4fa60efe9e79c9d71450a160c5072d1e4b4bb0"}
Jul 27 21:39:12.145520 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:12.145180    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:12.176676 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:12.176612    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp" podStartSLOduration=1.74234 podStartE2EDuration="6.176586757s" totalImagesPullingTime="3.385694695s" totalInitContainerRuntime="1.048552062s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:39:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:39:12.173507824 +0000 UTC m=+323.069317277" watchObservedRunningTime="2026-07-27 21:39:12.176586757 +0000 UTC m=+323.072396209"
Jul 27 21:39:18.123213 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:18.123184    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7c57b59594-lvwjn"
Jul 27 21:39:23.151307 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:23.151276    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d57c545b5-7pdmp"
Jul 27 21:39:36.033833 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.033791    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"]
Jul 27 21:39:36.042846 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.042824    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.045902 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.045689    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 27 21:39:36.045902 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.045707    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 27 21:39:36.045902 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.045762    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 21:39:36.045902 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.045826    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-p9txj\""
Jul 27 21:39:36.046312 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.045939    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 27 21:39:36.048297 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.048276    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"]
Jul 27 21:39:36.083250 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.083220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/971ff8b7-349e-4224-b9c5-95798d96ab1d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.083371 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.083348    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.083432 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.083397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.083495 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.083451    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.083558 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.083516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.083558 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.083535    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb2bz\" (UniqueName: \"kubernetes.io/projected/971ff8b7-349e-4224-b9c5-95798d96ab1d-kube-api-access-jb2bz\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.083651 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.083557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/971ff8b7-349e-4224-b9c5-95798d96ab1d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.184013 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.183981    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/971ff8b7-349e-4224-b9c5-95798d96ab1d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.184013 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.184020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.184251 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.184049    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.184251 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.184093    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.184251 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.184118    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.184251 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.184150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb2bz\" (UniqueName: \"kubernetes.io/projected/971ff8b7-349e-4224-b9c5-95798d96ab1d-kube-api-access-jb2bz\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.184251 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.184172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/971ff8b7-349e-4224-b9c5-95798d96ab1d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.185072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.185029    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.186631 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.186613    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.186706 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.186632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/971ff8b7-349e-4224-b9c5-95798d96ab1d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.186802 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.186786    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.186876 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.186857    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/971ff8b7-349e-4224-b9c5-95798d96ab1d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.192192 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.192170    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/971ff8b7-349e-4224-b9c5-95798d96ab1d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.192586 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.192564    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb2bz\" (UniqueName: \"kubernetes.io/projected/971ff8b7-349e-4224-b9c5-95798d96ab1d-kube-api-access-jb2bz\") pod \"istiod-openshift-gateway-5677ccb49b-l224d\" (UID: \"971ff8b7-349e-4224-b9c5-95798d96ab1d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.353660 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.353611    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:36.494536 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:36.494514    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"]
Jul 27 21:39:36.496765 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:39:36.496726    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971ff8b7_349e_4224_b9c5_95798d96ab1d.slice/crio-9f0c4c0e7266ee6cceeb3945662ce42d9ed65bfa8eff030acb326eabae0b1f0b WatchSource:0}: Error finding container 9f0c4c0e7266ee6cceeb3945662ce42d9ed65bfa8eff030acb326eabae0b1f0b: Status 404 returned error can't find the container with id 9f0c4c0e7266ee6cceeb3945662ce42d9ed65bfa8eff030acb326eabae0b1f0b
Jul 27 21:39:37.253717 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:37.253682    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d" event={"ID":"971ff8b7-349e-4224-b9c5-95798d96ab1d","Type":"ContainerStarted","Data":"9f0c4c0e7266ee6cceeb3945662ce42d9ed65bfa8eff030acb326eabae0b1f0b"}
Jul 27 21:39:38.649131 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:38.649073    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 21:39:38.649420 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:38.649159    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 21:39:39.266477 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:39.266438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d" event={"ID":"971ff8b7-349e-4224-b9c5-95798d96ab1d","Type":"ContainerStarted","Data":"e6c313041adb2e86bf8530694f656bbc1c8be66ed71de138744b2eddb9a83a10"}
Jul 27 21:39:39.266665 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:39.266638    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:39:39.268427 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:39.268401    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-l224d container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 21:39:39.268526 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:39.268444    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d" podUID="971ff8b7-349e-4224-b9c5-95798d96ab1d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 21:39:39.292210 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:39.292170    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d" podStartSLOduration=1.14230699 podStartE2EDuration="3.292156349s" totalImagesPullingTime="2.149849359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:39:39.290029771 +0000 UTC m=+350.185839225" watchObservedRunningTime="2026-07-27 21:39:39.292156349 +0000 UTC m=+350.187965778"
Jul 27 21:39:40.271166 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:39:40.271092    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-l224d"
Jul 27 21:40:47.910709 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:40:47.910676    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79976778f9-9tsft"]
Jul 27 21:41:12.937525 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:12.937428    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79976778f9-9tsft" podUID="0ec029cc-6372-4ccb-98b6-258c223baefa" containerName="console" containerID="cri-o://c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0" gracePeriod=15
Jul 27 21:41:13.201721 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.201702    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:41:13.384872 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.384846    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f4q8\" (UniqueName: \"kubernetes.io/projected/0ec029cc-6372-4ccb-98b6-258c223baefa-kube-api-access-4f4q8\") pod \"0ec029cc-6372-4ccb-98b6-258c223baefa\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") "
Jul 27 21:41:13.385027 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.384886    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-trusted-ca-bundle\") pod \"0ec029cc-6372-4ccb-98b6-258c223baefa\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") "
Jul 27 21:41:13.385027 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.384912    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-console-config\") pod \"0ec029cc-6372-4ccb-98b6-258c223baefa\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") "
Jul 27 21:41:13.385027 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.384947    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-oauth-config\") pod \"0ec029cc-6372-4ccb-98b6-258c223baefa\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") "
Jul 27 21:41:13.385027 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.384996    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-oauth-serving-cert\") pod \"0ec029cc-6372-4ccb-98b6-258c223baefa\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") "
Jul 27 21:41:13.385247 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.385035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-serving-cert\") pod \"0ec029cc-6372-4ccb-98b6-258c223baefa\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") "
Jul 27 21:41:13.385247 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.385088    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-service-ca\") pod \"0ec029cc-6372-4ccb-98b6-258c223baefa\" (UID: \"0ec029cc-6372-4ccb-98b6-258c223baefa\") "
Jul 27 21:41:13.385356 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.385276    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0ec029cc-6372-4ccb-98b6-258c223baefa" (UID: "0ec029cc-6372-4ccb-98b6-258c223baefa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:41:13.385415 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.385340    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-console-config" (OuterVolumeSpecName: "console-config") pod "0ec029cc-6372-4ccb-98b6-258c223baefa" (UID: "0ec029cc-6372-4ccb-98b6-258c223baefa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:41:13.385468 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.385442    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0ec029cc-6372-4ccb-98b6-258c223baefa" (UID: "0ec029cc-6372-4ccb-98b6-258c223baefa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:41:13.385539 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.385493    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-service-ca" (OuterVolumeSpecName: "service-ca") pod "0ec029cc-6372-4ccb-98b6-258c223baefa" (UID: "0ec029cc-6372-4ccb-98b6-258c223baefa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:41:13.387106 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.387080    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ec029cc-6372-4ccb-98b6-258c223baefa-kube-api-access-4f4q8" (OuterVolumeSpecName: "kube-api-access-4f4q8") pod "0ec029cc-6372-4ccb-98b6-258c223baefa" (UID: "0ec029cc-6372-4ccb-98b6-258c223baefa"). InnerVolumeSpecName "kube-api-access-4f4q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:41:13.387219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.387165    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0ec029cc-6372-4ccb-98b6-258c223baefa" (UID: "0ec029cc-6372-4ccb-98b6-258c223baefa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:41:13.387289 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.387270    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0ec029cc-6372-4ccb-98b6-258c223baefa" (UID: "0ec029cc-6372-4ccb-98b6-258c223baefa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:41:13.486480 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.486425    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-serving-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:41:13.486480 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.486447    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:41:13.486480 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.486456    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f4q8\" (UniqueName: \"kubernetes.io/projected/0ec029cc-6372-4ccb-98b6-258c223baefa-kube-api-access-4f4q8\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:41:13.486480 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.486464    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-trusted-ca-bundle\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:41:13.486480 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.486472    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-console-config\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:41:13.486480 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.486479    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ec029cc-6372-4ccb-98b6-258c223baefa-console-oauth-config\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:41:13.486811 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.486487    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ec029cc-6372-4ccb-98b6-258c223baefa-oauth-serving-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:41:13.673869 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.673844    2578 generic.go:356] "Generic (PLEG): container finished" podID="0ec029cc-6372-4ccb-98b6-258c223baefa" containerID="c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0" exitCode=0
Jul 27 21:41:13.673994 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.673893    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79976778f9-9tsft"
Jul 27 21:41:13.673994 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.673942    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79976778f9-9tsft" event={"ID":"0ec029cc-6372-4ccb-98b6-258c223baefa","Type":"ContainerDied","Data":"c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0"}
Jul 27 21:41:13.674092 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.673999    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79976778f9-9tsft" event={"ID":"0ec029cc-6372-4ccb-98b6-258c223baefa","Type":"ContainerDied","Data":"0673a0eb490f174f8c74154a436ee84e0f5324314373601f45c2121e19660899"}
Jul 27 21:41:13.674092 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.674026    2578 scope.go:117] "RemoveContainer" containerID="c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0"
Jul 27 21:41:13.682074 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.682054    2578 scope.go:117] "RemoveContainer" containerID="c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0"
Jul 27 21:41:13.682329 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:41:13.682312    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0\": container with ID starting with c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0 not found: ID does not exist" containerID="c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0"
Jul 27 21:41:13.682377 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.682337    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0"} err="failed to get container status \"c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0\": rpc error: code = NotFound desc = could not find container \"c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0\": container with ID starting with c513919d961803f1be76e7335d9e841a1bab8925245f6cec287570c09ff601a0 not found: ID does not exist"
Jul 27 21:41:13.701157 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.701140    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79976778f9-9tsft"]
Jul 27 21:41:13.707271 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:13.707250    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79976778f9-9tsft"]
Jul 27 21:41:15.605097 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:41:15.605050    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ec029cc-6372-4ccb-98b6-258c223baefa" path="/var/lib/kubelet/pods/0ec029cc-6372-4ccb-98b6-258c223baefa/volumes"
Jul 27 21:42:03.580268 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.580232    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-694cw"]
Jul 27 21:42:03.580708 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.580539    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ec029cc-6372-4ccb-98b6-258c223baefa" containerName="console"
Jul 27 21:42:03.580708 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.580550    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ec029cc-6372-4ccb-98b6-258c223baefa" containerName="console"
Jul 27 21:42:03.580708 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.580596    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ec029cc-6372-4ccb-98b6-258c223baefa" containerName="console"
Jul 27 21:42:03.583359 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.583342    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:03.586046 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.586027    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vx4tb\""
Jul 27 21:42:03.586269 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.586256    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 21:42:03.587222 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.587203    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 21:42:03.587308 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.587223    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 27 21:42:03.591184 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.591160    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-694cw"]
Jul 27 21:42:03.604678 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.604655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kncn\" (UniqueName: \"kubernetes.io/projected/65309e56-f41a-464e-b6a1-ababaf505a5b-kube-api-access-6kncn\") pod \"limitador-limitador-64c94fd64f-694cw\" (UID: \"65309e56-f41a-464e-b6a1-ababaf505a5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:03.604810 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.604714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65309e56-f41a-464e-b6a1-ababaf505a5b-config-file\") pod \"limitador-limitador-64c94fd64f-694cw\" (UID: \"65309e56-f41a-464e-b6a1-ababaf505a5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:03.675087 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.675053    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-694cw"]
Jul 27 21:42:03.705605 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.705584    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65309e56-f41a-464e-b6a1-ababaf505a5b-config-file\") pod \"limitador-limitador-64c94fd64f-694cw\" (UID: \"65309e56-f41a-464e-b6a1-ababaf505a5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:03.705702 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.705626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kncn\" (UniqueName: \"kubernetes.io/projected/65309e56-f41a-464e-b6a1-ababaf505a5b-kube-api-access-6kncn\") pod \"limitador-limitador-64c94fd64f-694cw\" (UID: \"65309e56-f41a-464e-b6a1-ababaf505a5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:03.706194 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.706177    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65309e56-f41a-464e-b6a1-ababaf505a5b-config-file\") pod \"limitador-limitador-64c94fd64f-694cw\" (UID: \"65309e56-f41a-464e-b6a1-ababaf505a5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:03.714867 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.714847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kncn\" (UniqueName: \"kubernetes.io/projected/65309e56-f41a-464e-b6a1-ababaf505a5b-kube-api-access-6kncn\") pod \"limitador-limitador-64c94fd64f-694cw\" (UID: \"65309e56-f41a-464e-b6a1-ababaf505a5b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:03.894299 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:03.894245    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:04.029449 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:04.029371    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-694cw"]
Jul 27 21:42:04.032043 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:42:04.032019    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65309e56_f41a_464e_b6a1_ababaf505a5b.slice/crio-a43bcd5e73181503a48680b760203aa0f7b21c6f7706f6fba50e6e4523252a4f WatchSource:0}: Error finding container a43bcd5e73181503a48680b760203aa0f7b21c6f7706f6fba50e6e4523252a4f: Status 404 returned error can't find the container with id a43bcd5e73181503a48680b760203aa0f7b21c6f7706f6fba50e6e4523252a4f
Jul 27 21:42:04.905603 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:04.897704    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw" event={"ID":"65309e56-f41a-464e-b6a1-ababaf505a5b","Type":"ContainerStarted","Data":"a43bcd5e73181503a48680b760203aa0f7b21c6f7706f6fba50e6e4523252a4f"}
Jul 27 21:42:06.908506 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:06.908472    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw" event={"ID":"65309e56-f41a-464e-b6a1-ababaf505a5b","Type":"ContainerStarted","Data":"8ff5f9420d242a5fb212632f0e59016fc723aab054418ccd13d28652d982a933"}
Jul 27 21:42:06.908861 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:06.908586    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:06.928155 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:06.928102    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw" podStartSLOduration=1.3288694030000001 podStartE2EDuration="3.928085854s" totalImagesPullingTime="2.599216451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:42:06.926030941 +0000 UTC m=+497.821840393" watchObservedRunningTime="2026-07-27 21:42:06.928085854 +0000 UTC m=+497.823895339"
Jul 27 21:42:17.914114 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:17.914085    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:19.477582 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:19.477543    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-694cw"]
Jul 27 21:42:19.478065 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:19.477814    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw" podUID="65309e56-f41a-464e-b6a1-ababaf505a5b" containerName="limitador" containerID="cri-o://8ff5f9420d242a5fb212632f0e59016fc723aab054418ccd13d28652d982a933" gracePeriod=30
Jul 27 21:42:19.971100 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:19.971068    2578 generic.go:356] "Generic (PLEG): container finished" podID="65309e56-f41a-464e-b6a1-ababaf505a5b" containerID="8ff5f9420d242a5fb212632f0e59016fc723aab054418ccd13d28652d982a933" exitCode=0
Jul 27 21:42:19.971212 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:19.971143    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw" event={"ID":"65309e56-f41a-464e-b6a1-ababaf505a5b","Type":"ContainerDied","Data":"8ff5f9420d242a5fb212632f0e59016fc723aab054418ccd13d28652d982a933"}
Jul 27 21:42:20.429091 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.429062    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:20.526353 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.526327    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kncn\" (UniqueName: \"kubernetes.io/projected/65309e56-f41a-464e-b6a1-ababaf505a5b-kube-api-access-6kncn\") pod \"65309e56-f41a-464e-b6a1-ababaf505a5b\" (UID: \"65309e56-f41a-464e-b6a1-ababaf505a5b\") "
Jul 27 21:42:20.526701 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.526445    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65309e56-f41a-464e-b6a1-ababaf505a5b-config-file\") pod \"65309e56-f41a-464e-b6a1-ababaf505a5b\" (UID: \"65309e56-f41a-464e-b6a1-ababaf505a5b\") "
Jul 27 21:42:20.526783 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.526762    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65309e56-f41a-464e-b6a1-ababaf505a5b-config-file" (OuterVolumeSpecName: "config-file") pod "65309e56-f41a-464e-b6a1-ababaf505a5b" (UID: "65309e56-f41a-464e-b6a1-ababaf505a5b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:42:20.528620 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.528600    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65309e56-f41a-464e-b6a1-ababaf505a5b-kube-api-access-6kncn" (OuterVolumeSpecName: "kube-api-access-6kncn") pod "65309e56-f41a-464e-b6a1-ababaf505a5b" (UID: "65309e56-f41a-464e-b6a1-ababaf505a5b"). InnerVolumeSpecName "kube-api-access-6kncn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:42:20.627374 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.627325    2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65309e56-f41a-464e-b6a1-ababaf505a5b-config-file\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:42:20.627374 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.627345    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kncn\" (UniqueName: \"kubernetes.io/projected/65309e56-f41a-464e-b6a1-ababaf505a5b-kube-api-access-6kncn\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:42:20.977178 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.977148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw" event={"ID":"65309e56-f41a-464e-b6a1-ababaf505a5b","Type":"ContainerDied","Data":"a43bcd5e73181503a48680b760203aa0f7b21c6f7706f6fba50e6e4523252a4f"}
Jul 27 21:42:20.977307 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.977183    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-694cw"
Jul 27 21:42:20.977307 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.977209    2578 scope.go:117] "RemoveContainer" containerID="8ff5f9420d242a5fb212632f0e59016fc723aab054418ccd13d28652d982a933"
Jul 27 21:42:20.999354 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:20.999333    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-694cw"]
Jul 27 21:42:21.008166 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:21.008146    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-694cw"]
Jul 27 21:42:21.605261 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:21.605231    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65309e56-f41a-464e-b6a1-ababaf505a5b" path="/var/lib/kubelet/pods/65309e56-f41a-464e-b6a1-ababaf505a5b/volumes"
Jul 27 21:42:22.444201 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.444170    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"]
Jul 27 21:42:22.444483 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.444469    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65309e56-f41a-464e-b6a1-ababaf505a5b" containerName="limitador"
Jul 27 21:42:22.444483 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.444484    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="65309e56-f41a-464e-b6a1-ababaf505a5b" containerName="limitador"
Jul 27 21:42:22.444565 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.444535    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="65309e56-f41a-464e-b6a1-ababaf505a5b" containerName="limitador"
Jul 27 21:42:22.449698 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.449678    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:22.452726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.452703    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-dbhm6\""
Jul 27 21:42:22.453292 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.453259    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 21:42:22.454496 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.454322    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 27 21:42:22.454496 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.454446    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 27 21:42:22.456239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.456217    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"]
Jul 27 21:42:22.542212 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.542188    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67jcr\" (UniqueName: \"kubernetes.io/projected/d5dc2039-640a-4377-8beb-fac18555ee1b-kube-api-access-67jcr\") pod \"postgres-69fb4594cf-ptpgp\" (UID: \"d5dc2039-640a-4377-8beb-fac18555ee1b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:22.542323 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.542230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d5dc2039-640a-4377-8beb-fac18555ee1b-data\") pod \"postgres-69fb4594cf-ptpgp\" (UID: \"d5dc2039-640a-4377-8beb-fac18555ee1b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:22.642799 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.642765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67jcr\" (UniqueName: \"kubernetes.io/projected/d5dc2039-640a-4377-8beb-fac18555ee1b-kube-api-access-67jcr\") pod \"postgres-69fb4594cf-ptpgp\" (UID: \"d5dc2039-640a-4377-8beb-fac18555ee1b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:22.643125 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.642815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d5dc2039-640a-4377-8beb-fac18555ee1b-data\") pod \"postgres-69fb4594cf-ptpgp\" (UID: \"d5dc2039-640a-4377-8beb-fac18555ee1b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:22.643125 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.643088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d5dc2039-640a-4377-8beb-fac18555ee1b-data\") pod \"postgres-69fb4594cf-ptpgp\" (UID: \"d5dc2039-640a-4377-8beb-fac18555ee1b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:22.655944 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.655920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67jcr\" (UniqueName: \"kubernetes.io/projected/d5dc2039-640a-4377-8beb-fac18555ee1b-kube-api-access-67jcr\") pod \"postgres-69fb4594cf-ptpgp\" (UID: \"d5dc2039-640a-4377-8beb-fac18555ee1b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:22.760920 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.760852    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:22.901199 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.901169    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"]
Jul 27 21:42:22.903131 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:42:22.903105    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5dc2039_640a_4377_8beb_fac18555ee1b.slice/crio-b4c08e68c32e059aa8fa54b9e0f873d380c1ae69c7363ab803d38453ba67037b WatchSource:0}: Error finding container b4c08e68c32e059aa8fa54b9e0f873d380c1ae69c7363ab803d38453ba67037b: Status 404 returned error can't find the container with id b4c08e68c32e059aa8fa54b9e0f873d380c1ae69c7363ab803d38453ba67037b
Jul 27 21:42:22.988381 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:22.988345    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp" event={"ID":"d5dc2039-640a-4377-8beb-fac18555ee1b","Type":"ContainerStarted","Data":"b4c08e68c32e059aa8fa54b9e0f873d380c1ae69c7363ab803d38453ba67037b"}
Jul 27 21:42:28.016595 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:28.016561    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp" event={"ID":"d5dc2039-640a-4377-8beb-fac18555ee1b","Type":"ContainerStarted","Data":"221da985d554009d03fcf1f00926d3a7a80132f9dc415f41195725f9831f3dbb"}
Jul 27 21:42:28.016963 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:28.016798    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:28.036223 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:28.036179    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp" podStartSLOduration=1.040182615 podStartE2EDuration="6.036165521s" totalImagesPullingTime="4.995982906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:42:28.034232322 +0000 UTC m=+518.930041773" watchObservedRunningTime="2026-07-27 21:42:28.036165521 +0000 UTC m=+518.931974971"
Jul 27 21:42:34.051723 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:34.051687    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptpgp"
Jul 27 21:42:41.479298 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.479219    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84796c7b49-7fvt4"]
Jul 27 21:42:41.482553 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.482537    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:41.486042 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.486021    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 21:42:41.486275 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.486256    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wbhc9\""
Jul 27 21:42:41.487188 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.487169    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 21:42:41.495161 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.495138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84796c7b49-7fvt4"]
Jul 27 21:42:41.586486 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.586456    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3843a930-01c6-4bb2-ae73-2ce5904b475b-webhook-cert\") pod \"maas-controller-84796c7b49-7fvt4\" (UID: \"3843a930-01c6-4bb2-ae73-2ce5904b475b\") " pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:41.586605 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.586489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqm7f\" (UniqueName: \"kubernetes.io/projected/3843a930-01c6-4bb2-ae73-2ce5904b475b-kube-api-access-kqm7f\") pod \"maas-controller-84796c7b49-7fvt4\" (UID: \"3843a930-01c6-4bb2-ae73-2ce5904b475b\") " pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:41.653324 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.653290    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-f4926"]
Jul 27 21:42:41.656635 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.656613    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:41.667016 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.666992    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-f4926"]
Jul 27 21:42:41.687079 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.687058    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3843a930-01c6-4bb2-ae73-2ce5904b475b-webhook-cert\") pod \"maas-controller-84796c7b49-7fvt4\" (UID: \"3843a930-01c6-4bb2-ae73-2ce5904b475b\") " pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:41.687168 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.687091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqm7f\" (UniqueName: \"kubernetes.io/projected/3843a930-01c6-4bb2-ae73-2ce5904b475b-kube-api-access-kqm7f\") pod \"maas-controller-84796c7b49-7fvt4\" (UID: \"3843a930-01c6-4bb2-ae73-2ce5904b475b\") " pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:41.689540 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.689517    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3843a930-01c6-4bb2-ae73-2ce5904b475b-webhook-cert\") pod \"maas-controller-84796c7b49-7fvt4\" (UID: \"3843a930-01c6-4bb2-ae73-2ce5904b475b\") " pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:41.696088 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.696059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqm7f\" (UniqueName: \"kubernetes.io/projected/3843a930-01c6-4bb2-ae73-2ce5904b475b-kube-api-access-kqm7f\") pod \"maas-controller-84796c7b49-7fvt4\" (UID: \"3843a930-01c6-4bb2-ae73-2ce5904b475b\") " pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:41.787842 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.787790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2b89f25-bd65-4862-aacc-cce61799597f-webhook-cert\") pod \"maas-controller-6fbcf7897d-f4926\" (UID: \"d2b89f25-bd65-4862-aacc-cce61799597f\") " pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:41.787842 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.787822    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfr5k\" (UniqueName: \"kubernetes.io/projected/d2b89f25-bd65-4862-aacc-cce61799597f-kube-api-access-hfr5k\") pod \"maas-controller-6fbcf7897d-f4926\" (UID: \"d2b89f25-bd65-4862-aacc-cce61799597f\") " pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:41.792757 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.792721    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:41.888266 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.888237    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2b89f25-bd65-4862-aacc-cce61799597f-webhook-cert\") pod \"maas-controller-6fbcf7897d-f4926\" (UID: \"d2b89f25-bd65-4862-aacc-cce61799597f\") " pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:41.888433 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.888273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfr5k\" (UniqueName: \"kubernetes.io/projected/d2b89f25-bd65-4862-aacc-cce61799597f-kube-api-access-hfr5k\") pod \"maas-controller-6fbcf7897d-f4926\" (UID: \"d2b89f25-bd65-4862-aacc-cce61799597f\") " pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:41.896955 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.896930    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2b89f25-bd65-4862-aacc-cce61799597f-webhook-cert\") pod \"maas-controller-6fbcf7897d-f4926\" (UID: \"d2b89f25-bd65-4862-aacc-cce61799597f\") " pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:41.900455 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.900396    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfr5k\" (UniqueName: \"kubernetes.io/projected/d2b89f25-bd65-4862-aacc-cce61799597f-kube-api-access-hfr5k\") pod \"maas-controller-6fbcf7897d-f4926\" (UID: \"d2b89f25-bd65-4862-aacc-cce61799597f\") " pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:41.956838 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.956702    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84796c7b49-7fvt4"]
Jul 27 21:42:41.958963 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:42:41.958937    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3843a930_01c6_4bb2_ae73_2ce5904b475b.slice/crio-e5548aa1f43b2a8c633a079560c4070c7af7d4e38eb4a78932d76c0a20381b8c WatchSource:0}: Error finding container e5548aa1f43b2a8c633a079560c4070c7af7d4e38eb4a78932d76c0a20381b8c: Status 404 returned error can't find the container with id e5548aa1f43b2a8c633a079560c4070c7af7d4e38eb4a78932d76c0a20381b8c
Jul 27 21:42:41.967508 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:41.967489    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:42.079316 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:42.079279    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84796c7b49-7fvt4" event={"ID":"3843a930-01c6-4bb2-ae73-2ce5904b475b","Type":"ContainerStarted","Data":"e5548aa1f43b2a8c633a079560c4070c7af7d4e38eb4a78932d76c0a20381b8c"}
Jul 27 21:42:42.102546 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:42.102524    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-f4926"]
Jul 27 21:42:42.104822 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:42:42.104799    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b89f25_bd65_4862_aacc_cce61799597f.slice/crio-e029dcc0c0b083a053ae2d3e2b9978debbc891996f60463a0350bbf710860795 WatchSource:0}: Error finding container e029dcc0c0b083a053ae2d3e2b9978debbc891996f60463a0350bbf710860795: Status 404 returned error can't find the container with id e029dcc0c0b083a053ae2d3e2b9978debbc891996f60463a0350bbf710860795
Jul 27 21:42:43.087878 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:43.087843    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-f4926" event={"ID":"d2b89f25-bd65-4862-aacc-cce61799597f","Type":"ContainerStarted","Data":"e029dcc0c0b083a053ae2d3e2b9978debbc891996f60463a0350bbf710860795"}
Jul 27 21:42:45.098155 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:45.098117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84796c7b49-7fvt4" event={"ID":"3843a930-01c6-4bb2-ae73-2ce5904b475b","Type":"ContainerStarted","Data":"ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5"}
Jul 27 21:42:45.098614 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:45.098195    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:45.099573 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:45.099553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-f4926" event={"ID":"d2b89f25-bd65-4862-aacc-cce61799597f","Type":"ContainerStarted","Data":"86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036"}
Jul 27 21:42:45.099681 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:45.099661    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:45.137838 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:45.137801    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84796c7b49-7fvt4" podStartSLOduration=1.702097068 podStartE2EDuration="4.13778961s" totalImagesPullingTime="2.435692542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:42:45.136202669 +0000 UTC m=+536.032012121" watchObservedRunningTime="2026-07-27 21:42:45.13778961 +0000 UTC m=+536.033599062"
Jul 27 21:42:45.170557 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:45.170511    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fbcf7897d-f4926" podStartSLOduration=1.873760501 podStartE2EDuration="4.170497612s" totalImagesPullingTime="2.296737111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:42:45.168891479 +0000 UTC m=+536.064700943" watchObservedRunningTime="2026-07-27 21:42:45.170497612 +0000 UTC m=+536.066307081"
Jul 27 21:42:56.107158 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.107120    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:56.107563 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.107188    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:42:56.175863 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.175832    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84796c7b49-7fvt4"]
Jul 27 21:42:56.176041 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.176017    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84796c7b49-7fvt4" podUID="3843a930-01c6-4bb2-ae73-2ce5904b475b" containerName="manager" containerID="cri-o://ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5" gracePeriod=30
Jul 27 21:42:56.420470 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.420450    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:56.500604 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.500580    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqm7f\" (UniqueName: \"kubernetes.io/projected/3843a930-01c6-4bb2-ae73-2ce5904b475b-kube-api-access-kqm7f\") pod \"3843a930-01c6-4bb2-ae73-2ce5904b475b\" (UID: \"3843a930-01c6-4bb2-ae73-2ce5904b475b\") "
Jul 27 21:42:56.500717 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.500631    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3843a930-01c6-4bb2-ae73-2ce5904b475b-webhook-cert\") pod \"3843a930-01c6-4bb2-ae73-2ce5904b475b\" (UID: \"3843a930-01c6-4bb2-ae73-2ce5904b475b\") "
Jul 27 21:42:56.502849 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.502820    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3843a930-01c6-4bb2-ae73-2ce5904b475b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3843a930-01c6-4bb2-ae73-2ce5904b475b" (UID: "3843a930-01c6-4bb2-ae73-2ce5904b475b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:42:56.502943 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.502871    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3843a930-01c6-4bb2-ae73-2ce5904b475b-kube-api-access-kqm7f" (OuterVolumeSpecName: "kube-api-access-kqm7f") pod "3843a930-01c6-4bb2-ae73-2ce5904b475b" (UID: "3843a930-01c6-4bb2-ae73-2ce5904b475b"). InnerVolumeSpecName "kube-api-access-kqm7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:42:56.601741 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.601719    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqm7f\" (UniqueName: \"kubernetes.io/projected/3843a930-01c6-4bb2-ae73-2ce5904b475b-kube-api-access-kqm7f\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:42:56.601863 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:56.601748    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3843a930-01c6-4bb2-ae73-2ce5904b475b-webhook-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:42:57.157860 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.157830    2578 generic.go:356] "Generic (PLEG): container finished" podID="3843a930-01c6-4bb2-ae73-2ce5904b475b" containerID="ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5" exitCode=0
Jul 27 21:42:57.158213 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.157877    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84796c7b49-7fvt4"
Jul 27 21:42:57.158213 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.157913    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84796c7b49-7fvt4" event={"ID":"3843a930-01c6-4bb2-ae73-2ce5904b475b","Type":"ContainerDied","Data":"ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5"}
Jul 27 21:42:57.158213 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.157965    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84796c7b49-7fvt4" event={"ID":"3843a930-01c6-4bb2-ae73-2ce5904b475b","Type":"ContainerDied","Data":"e5548aa1f43b2a8c633a079560c4070c7af7d4e38eb4a78932d76c0a20381b8c"}
Jul 27 21:42:57.158213 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.157988    2578 scope.go:117] "RemoveContainer" containerID="ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5"
Jul 27 21:42:57.166451 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.166434    2578 scope.go:117] "RemoveContainer" containerID="ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5"
Jul 27 21:42:57.166699 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:42:57.166680    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5\": container with ID starting with ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5 not found: ID does not exist" containerID="ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5"
Jul 27 21:42:57.166798 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.166711    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5"} err="failed to get container status \"ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5\": rpc error: code = NotFound desc = could not find container \"ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5\": container with ID starting with ba98da088f0aaf4fa412a6aac963ad7d8f8e4017d7d2c2954faeafc424bf39d5 not found: ID does not exist"
Jul 27 21:42:57.189565 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.189534    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84796c7b49-7fvt4"]
Jul 27 21:42:57.198303 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.198285    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84796c7b49-7fvt4"]
Jul 27 21:42:57.604414 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:42:57.604359    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3843a930-01c6-4bb2-ae73-2ce5904b475b" path="/var/lib/kubelet/pods/3843a930-01c6-4bb2-ae73-2ce5904b475b/volumes"
Jul 27 21:43:16.814209 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.814169    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"]
Jul 27 21:43:16.814787 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.814615    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3843a930-01c6-4bb2-ae73-2ce5904b475b" containerName="manager"
Jul 27 21:43:16.814787 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.814634    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3843a930-01c6-4bb2-ae73-2ce5904b475b" containerName="manager"
Jul 27 21:43:16.814787 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.814721    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3843a930-01c6-4bb2-ae73-2ce5904b475b" containerName="manager"
Jul 27 21:43:16.819095 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.819073    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:16.822091 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.822069    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pj4wj\""
Jul 27 21:43:16.822278 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.822261    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 27 21:43:16.832042 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.832022    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"]
Jul 27 21:43:16.866154 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.866123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8a613c4-92d9-4301-a365-f4df038b5b5b-maas-api-tls\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:16.866311 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.866166    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s67r6\" (UniqueName: \"kubernetes.io/projected/d8a613c4-92d9-4301-a365-f4df038b5b5b-kube-api-access-s67r6\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:16.866311 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.866264    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8a613c4-92d9-4301-a365-f4df038b5b5b-openshift-service-ca\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:16.966943 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.966911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8a613c4-92d9-4301-a365-f4df038b5b5b-openshift-service-ca\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:16.967112 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.966954    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8a613c4-92d9-4301-a365-f4df038b5b5b-maas-api-tls\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:16.967112 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.966975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s67r6\" (UniqueName: \"kubernetes.io/projected/d8a613c4-92d9-4301-a365-f4df038b5b5b-kube-api-access-s67r6\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:16.967112 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:43:16.967078    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 27 21:43:16.967245 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:43:16.967146    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8a613c4-92d9-4301-a365-f4df038b5b5b-maas-api-tls podName:d8a613c4-92d9-4301-a365-f4df038b5b5b nodeName:}" failed. No retries permitted until 2026-07-27 21:43:17.467125804 +0000 UTC m=+568.362935233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d8a613c4-92d9-4301-a365-f4df038b5b5b-maas-api-tls") pod "maas-api-bc9f765f9-lm6bf" (UID: "d8a613c4-92d9-4301-a365-f4df038b5b5b") : secret "maas-api-serving-cert" not found
Jul 27 21:43:16.967495 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.967477    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8a613c4-92d9-4301-a365-f4df038b5b5b-openshift-service-ca\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:16.976282 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:16.976254    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s67r6\" (UniqueName: \"kubernetes.io/projected/d8a613c4-92d9-4301-a365-f4df038b5b5b-kube-api-access-s67r6\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:17.397388 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.397353    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-885jp"]
Jul 27 21:43:17.399819 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.399802    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:17.402436 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.402406    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 27 21:43:17.402665 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.402649    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9c4jr\""
Jul 27 21:43:17.413012 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.412989    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-885jp"]
Jul 27 21:43:17.470686 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.470650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js9gj\" (UniqueName: \"kubernetes.io/projected/05798666-dd9a-440c-98c8-280baf18deb3-kube-api-access-js9gj\") pod \"payload-processing-5cc6cd444b-885jp\" (UID: \"05798666-dd9a-440c-98c8-280baf18deb3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:17.470876 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.470704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8a613c4-92d9-4301-a365-f4df038b5b5b-maas-api-tls\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:17.470876 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.470794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05798666-dd9a-440c-98c8-280baf18deb3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-885jp\" (UID: \"05798666-dd9a-440c-98c8-280baf18deb3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:17.473411 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.473386    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8a613c4-92d9-4301-a365-f4df038b5b5b-maas-api-tls\") pod \"maas-api-bc9f765f9-lm6bf\" (UID: \"d8a613c4-92d9-4301-a365-f4df038b5b5b\") " pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:17.571637 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.571595    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05798666-dd9a-440c-98c8-280baf18deb3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-885jp\" (UID: \"05798666-dd9a-440c-98c8-280baf18deb3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:17.571851 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.571693    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js9gj\" (UniqueName: \"kubernetes.io/projected/05798666-dd9a-440c-98c8-280baf18deb3-kube-api-access-js9gj\") pod \"payload-processing-5cc6cd444b-885jp\" (UID: \"05798666-dd9a-440c-98c8-280baf18deb3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:17.572367 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.572342    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05798666-dd9a-440c-98c8-280baf18deb3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-885jp\" (UID: \"05798666-dd9a-440c-98c8-280baf18deb3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:17.582106 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.582084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js9gj\" (UniqueName: \"kubernetes.io/projected/05798666-dd9a-440c-98c8-280baf18deb3-kube-api-access-js9gj\") pod \"payload-processing-5cc6cd444b-885jp\" (UID: \"05798666-dd9a-440c-98c8-280baf18deb3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:17.710760 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.710709    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:17.728885 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.728857    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:17.856858 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.856831    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-885jp"]
Jul 27 21:43:17.860023 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:43:17.859996    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05798666_dd9a_440c_98c8_280baf18deb3.slice/crio-a181916912958608d9629b7adba96fac3be981d69229dcd3430ff7cba821ac57 WatchSource:0}: Error finding container a181916912958608d9629b7adba96fac3be981d69229dcd3430ff7cba821ac57: Status 404 returned error can't find the container with id a181916912958608d9629b7adba96fac3be981d69229dcd3430ff7cba821ac57
Jul 27 21:43:17.883599 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:17.883575    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"]
Jul 27 21:43:17.886053 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:43:17.886028    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a613c4_92d9_4301_a365_f4df038b5b5b.slice/crio-bd3c41857ac657d36b763af2a9a68a030eb9a1b5dfca577efca8766df72cf4a8 WatchSource:0}: Error finding container bd3c41857ac657d36b763af2a9a68a030eb9a1b5dfca577efca8766df72cf4a8: Status 404 returned error can't find the container with id bd3c41857ac657d36b763af2a9a68a030eb9a1b5dfca577efca8766df72cf4a8
Jul 27 21:43:18.248868 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:18.248831    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf" event={"ID":"d8a613c4-92d9-4301-a365-f4df038b5b5b","Type":"ContainerStarted","Data":"bd3c41857ac657d36b763af2a9a68a030eb9a1b5dfca577efca8766df72cf4a8"}
Jul 27 21:43:18.249954 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:18.249931    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-885jp" event={"ID":"05798666-dd9a-440c-98c8-280baf18deb3","Type":"ContainerStarted","Data":"a181916912958608d9629b7adba96fac3be981d69229dcd3430ff7cba821ac57"}
Jul 27 21:43:20.265448 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:20.265415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf" event={"ID":"d8a613c4-92d9-4301-a365-f4df038b5b5b","Type":"ContainerStarted","Data":"e204bee3f549377cc3788994a51c7b547d0db47d6221928016781619352f228c"}
Jul 27 21:43:20.265896 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:20.265495    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:20.267356 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:20.267326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-885jp" event={"ID":"05798666-dd9a-440c-98c8-280baf18deb3","Type":"ContainerStarted","Data":"209e3c4108048e8f8b44c1e09bacf94f27f28d730f4e3e0c000ee24a9bd451ad"}
Jul 27 21:43:20.267491 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:20.267444    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:20.285072 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:20.285018    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf" podStartSLOduration=2.44876058 podStartE2EDuration="4.28499772s" totalImagesPullingTime="1.83623714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:43:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:43:20.283966848 +0000 UTC m=+571.179776299" watchObservedRunningTime="2026-07-27 21:43:20.28499772 +0000 UTC m=+571.180807174"
Jul 27 21:43:20.301428 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:20.301382    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-885jp" podStartSLOduration=1.434707224 podStartE2EDuration="3.301361618s" totalImagesPullingTime="1.866654394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:43:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:43:20.300986327 +0000 UTC m=+571.196795785" watchObservedRunningTime="2026-07-27 21:43:20.301361618 +0000 UTC m=+571.197171070"
Jul 27 21:43:26.280651 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:26.280623    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-bc9f765f9-lm6bf"
Jul 27 21:43:31.276769 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:31.276722    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-885jp"
Jul 27 21:43:47.083497 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.083463    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"]
Jul 27 21:43:47.086796 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.086771    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.090190 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.090165    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 21:43:47.090329 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.090310    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dwq99\""
Jul 27 21:43:47.091384 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.091359    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 21:43:47.091477 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.091401    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 27 21:43:47.102464 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.102440    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"]
Jul 27 21:43:47.211936 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.211907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.211936 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.211936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.212119 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.211966    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.212119 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.212012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/060c5768-72ca-4bd5-bcbe-935f60768fee-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.212182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.212126    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.212182 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.212145    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpw9k\" (UniqueName: \"kubernetes.io/projected/060c5768-72ca-4bd5-bcbe-935f60768fee-kube-api-access-vpw9k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.313191 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.313163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.313323 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.313208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.313323 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.313240    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.313323 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.313265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/060c5768-72ca-4bd5-bcbe-935f60768fee-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.313483 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.313334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.313483 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.313453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpw9k\" (UniqueName: \"kubernetes.io/projected/060c5768-72ca-4bd5-bcbe-935f60768fee-kube-api-access-vpw9k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.313606 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.313585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.313668 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.313654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.313719 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.313694    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.315575 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.315551    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/060c5768-72ca-4bd5-bcbe-935f60768fee-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.315834 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.315820    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/060c5768-72ca-4bd5-bcbe-935f60768fee-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.326504 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.326476    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpw9k\" (UniqueName: \"kubernetes.io/projected/060c5768-72ca-4bd5-bcbe-935f60768fee-kube-api-access-vpw9k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dkdkc\" (UID: \"060c5768-72ca-4bd5-bcbe-935f60768fee\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.397576 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.397519    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:47.530321 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:47.530289    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"]
Jul 27 21:43:47.533335 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:43:47.533311    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060c5768_72ca_4bd5_bcbe_935f60768fee.slice/crio-a854759bf57aee026fb21f4b6e73a2869d5ffd4107bec2329b6799d7f160822c WatchSource:0}: Error finding container a854759bf57aee026fb21f4b6e73a2869d5ffd4107bec2329b6799d7f160822c: Status 404 returned error can't find the container with id a854759bf57aee026fb21f4b6e73a2869d5ffd4107bec2329b6799d7f160822c
Jul 27 21:43:48.401645 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:48.401600    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc" event={"ID":"060c5768-72ca-4bd5-bcbe-935f60768fee","Type":"ContainerStarted","Data":"a854759bf57aee026fb21f4b6e73a2869d5ffd4107bec2329b6799d7f160822c"}
Jul 27 21:43:49.407471 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:49.407435    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc" event={"ID":"060c5768-72ca-4bd5-bcbe-935f60768fee","Type":"ContainerStarted","Data":"7388fc301c1da74c4ac68418e9a2c88e05672b4aee1f71671538924da831f586"}
Jul 27 21:43:49.428000 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:49.427955    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc" podStartSLOduration=1.487230686 podStartE2EDuration="2.427940495s" totalImagesPullingTime="940.709809ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:43:49.424951756 +0000 UTC m=+600.320761209" watchObservedRunningTime="2026-07-27 21:43:49.427940495 +0000 UTC m=+600.323749947"
Jul 27 21:43:49.475728 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:49.475701    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:43:49.477248 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:49.477229    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:43:57.398726 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:57.398688    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:57.399189 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:57.398775    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:57.411302 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:57.411270    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:57.456179 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:57.456147    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dkdkc"
Jul 27 21:43:59.172774 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.172715    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"]
Jul 27 21:43:59.178347 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.178329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.181143 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.181100    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 27 21:43:59.186618 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.186560    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"]
Jul 27 21:43:59.305525 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.305502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.305644 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.305534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.305644 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.305569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.305644 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.305623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.305789 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.305648    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr5vw\" (UniqueName: \"kubernetes.io/projected/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-kube-api-access-jr5vw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.305789 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.305682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.406558 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.406525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.406668 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.406559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.406715 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.406693    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.406785 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.406771    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.406825 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.406813    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr5vw\" (UniqueName: \"kubernetes.io/projected/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-kube-api-access-jr5vw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.406880 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.406843    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.406934 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.406888    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.406986 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.406951    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.407039 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.407012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.409219 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.409202    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.409393 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.409375    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.415682 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.415664    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr5vw\" (UniqueName: \"kubernetes.io/projected/ef18c4b1-db1f-48d5-8dbf-0cb255c7465d-kube-api-access-jr5vw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rttgj\" (UID: \"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.490568 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.490506    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:43:59.625716 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:43:59.625694    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"]
Jul 27 21:43:59.628224 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:43:59.628191    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef18c4b1_db1f_48d5_8dbf_0cb255c7465d.slice/crio-69d238d2f9e5775f1ddb64287b279929b5713236e63b68a25723e4343f8fde0f WatchSource:0}: Error finding container 69d238d2f9e5775f1ddb64287b279929b5713236e63b68a25723e4343f8fde0f: Status 404 returned error can't find the container with id 69d238d2f9e5775f1ddb64287b279929b5713236e63b68a25723e4343f8fde0f
Jul 27 21:44:00.459713 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:44:00.459679    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj" event={"ID":"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d","Type":"ContainerStarted","Data":"53a68972694bd1145c40711d9e571bfde813303455ed7be851db0fab8c58cfa3"}
Jul 27 21:44:00.460097 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:44:00.459727    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj" event={"ID":"ef18c4b1-db1f-48d5-8dbf-0cb255c7465d","Type":"ContainerStarted","Data":"69d238d2f9e5775f1ddb64287b279929b5713236e63b68a25723e4343f8fde0f"}
Jul 27 21:44:00.487123 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:44:00.487071    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj" podStartSLOduration=1.321207082 podStartE2EDuration="1.487054546s" totalImagesPullingTime="165.847464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:44:00.4856631 +0000 UTC m=+611.381472562" watchObservedRunningTime="2026-07-27 21:44:00.487054546 +0000 UTC m=+611.382864063"
Jul 27 21:44:09.491702 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:44:09.491661    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:44:09.494216 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:44:09.491756    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:44:09.505231 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:44:09.505204    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:44:09.516895 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:44:09.516876    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rttgj"
Jul 27 21:45:00.147173 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:00.147140    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6"]
Jul 27 21:45:00.150569 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:00.150552    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6"
Jul 27 21:45:00.153892 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:00.153869    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxwpp\""
Jul 27 21:45:00.160239 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:00.160216    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6"]
Jul 27 21:45:00.161562 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:00.161544    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6"
Jul 27 21:45:00.300882 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:00.300859    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6"]
Jul 27 21:45:00.302205 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:45:00.302174    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b521213_1c52_4635_8277_981411b3f138.slice/crio-cf779f2b54dc139a490c442b82994c6437f7d1acb266d023d1ae38a087439922 WatchSource:0}: Error finding container cf779f2b54dc139a490c442b82994c6437f7d1acb266d023d1ae38a087439922: Status 404 returned error can't find the container with id cf779f2b54dc139a490c442b82994c6437f7d1acb266d023d1ae38a087439922
Jul 27 21:45:00.304292 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:00.304274    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:45:00.743599 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:00.743565    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6" event={"ID":"0b521213-1c52-4635-8277-981411b3f138","Type":"ContainerStarted","Data":"cf779f2b54dc139a490c442b82994c6437f7d1acb266d023d1ae38a087439922"}
Jul 27 21:45:03.759438 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:03.759410    2578 generic.go:356] "Generic (PLEG): container finished" podID="0b521213-1c52-4635-8277-981411b3f138" containerID="b53ad5c9efbe8ad60158643dff7596889f45ac927342898106c51e7067747bf0" exitCode=0
Jul 27 21:45:03.759757 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:03.759493    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6" event={"ID":"0b521213-1c52-4635-8277-981411b3f138","Type":"ContainerDied","Data":"b53ad5c9efbe8ad60158643dff7596889f45ac927342898106c51e7067747bf0"}
Jul 27 21:45:04.898355 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:04.898334    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6"
Jul 27 21:45:05.769397 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:05.769370    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6" event={"ID":"0b521213-1c52-4635-8277-981411b3f138","Type":"ContainerDied","Data":"cf779f2b54dc139a490c442b82994c6437f7d1acb266d023d1ae38a087439922"}
Jul 27 21:45:05.769531 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:05.769410    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-wzml6"
Jul 27 21:45:05.769531 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:45:05.769413    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf779f2b54dc139a490c442b82994c6437f7d1acb266d023d1ae38a087439922"
Jul 27 21:46:02.244627 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:02.244550    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-f4926"]
Jul 27 21:46:02.245104 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:02.244758    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6fbcf7897d-f4926" podUID="d2b89f25-bd65-4862-aacc-cce61799597f" containerName="manager" containerID="cri-o://86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036" gracePeriod=30
Jul 27 21:46:02.498410 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:02.498359    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:46:02.596611 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:02.596580    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2b89f25-bd65-4862-aacc-cce61799597f-webhook-cert\") pod \"d2b89f25-bd65-4862-aacc-cce61799597f\" (UID: \"d2b89f25-bd65-4862-aacc-cce61799597f\") "
Jul 27 21:46:02.596781 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:02.596650    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfr5k\" (UniqueName: \"kubernetes.io/projected/d2b89f25-bd65-4862-aacc-cce61799597f-kube-api-access-hfr5k\") pod \"d2b89f25-bd65-4862-aacc-cce61799597f\" (UID: \"d2b89f25-bd65-4862-aacc-cce61799597f\") "
Jul 27 21:46:02.598875 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:02.598850    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2b89f25-bd65-4862-aacc-cce61799597f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d2b89f25-bd65-4862-aacc-cce61799597f" (UID: "d2b89f25-bd65-4862-aacc-cce61799597f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:46:02.598940 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:02.598921    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2b89f25-bd65-4862-aacc-cce61799597f-kube-api-access-hfr5k" (OuterVolumeSpecName: "kube-api-access-hfr5k") pod "d2b89f25-bd65-4862-aacc-cce61799597f" (UID: "d2b89f25-bd65-4862-aacc-cce61799597f"). InnerVolumeSpecName "kube-api-access-hfr5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:46:02.698096 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:02.698069    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfr5k\" (UniqueName: \"kubernetes.io/projected/d2b89f25-bd65-4862-aacc-cce61799597f-kube-api-access-hfr5k\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:46:02.698096 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:02.698094    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2b89f25-bd65-4862-aacc-cce61799597f-webhook-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:46:03.029565 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.029530    2578 generic.go:356] "Generic (PLEG): container finished" podID="d2b89f25-bd65-4862-aacc-cce61799597f" containerID="86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036" exitCode=0
Jul 27 21:46:03.029699 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.029581    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-f4926"
Jul 27 21:46:03.029699 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.029620    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-f4926" event={"ID":"d2b89f25-bd65-4862-aacc-cce61799597f","Type":"ContainerDied","Data":"86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036"}
Jul 27 21:46:03.029699 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.029671    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-f4926" event={"ID":"d2b89f25-bd65-4862-aacc-cce61799597f","Type":"ContainerDied","Data":"e029dcc0c0b083a053ae2d3e2b9978debbc891996f60463a0350bbf710860795"}
Jul 27 21:46:03.029699 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.029693    2578 scope.go:117] "RemoveContainer" containerID="86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036"
Jul 27 21:46:03.039645 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.039628    2578 scope.go:117] "RemoveContainer" containerID="86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036"
Jul 27 21:46:03.039948 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:46:03.039928    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036\": container with ID starting with 86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036 not found: ID does not exist" containerID="86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036"
Jul 27 21:46:03.040028 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.039961    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036"} err="failed to get container status \"86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036\": rpc error: code = NotFound desc = could not find container \"86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036\": container with ID starting with 86aadd8facc68a03d9b73fbb66d2870d9f2756def0c6d5a8112c314d3f3a7036 not found: ID does not exist"
Jul 27 21:46:03.052824 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.052799    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-f4926"]
Jul 27 21:46:03.056995 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.056976    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-f4926"]
Jul 27 21:46:03.605176 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:46:03.605147    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2b89f25-bd65-4862-aacc-cce61799597f" path="/var/lib/kubelet/pods/d2b89f25-bd65-4862-aacc-cce61799597f/volumes"
Jul 27 21:47:03.902560 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.902523    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-ccsgr"]
Jul 27 21:47:03.903055 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.902872    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b89f25-bd65-4862-aacc-cce61799597f" containerName="manager"
Jul 27 21:47:03.903055 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.902883    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b89f25-bd65-4862-aacc-cce61799597f" containerName="manager"
Jul 27 21:47:03.903055 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.902902    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b521213-1c52-4635-8277-981411b3f138" containerName="cleanup"
Jul 27 21:47:03.903055 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.902907    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b521213-1c52-4635-8277-981411b3f138" containerName="cleanup"
Jul 27 21:47:03.903055 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.902957    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b521213-1c52-4635-8277-981411b3f138" containerName="cleanup"
Jul 27 21:47:03.903055 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.902968    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b89f25-bd65-4862-aacc-cce61799597f" containerName="manager"
Jul 27 21:47:03.907181 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.907165    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:03.911325 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.911290    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wbhc9\""
Jul 27 21:47:03.911523 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.911329    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 21:47:03.911523 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.911292    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 21:47:03.915333 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:03.915312    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-ccsgr"]
Jul 27 21:47:04.060873 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:04.060846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/345ac969-6ff6-4204-954a-ae927ca88b00-webhook-cert\") pod \"maas-controller-6fbcf7897d-ccsgr\" (UID: \"345ac969-6ff6-4204-954a-ae927ca88b00\") " pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:04.061039 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:04.060882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdb5b\" (UniqueName: \"kubernetes.io/projected/345ac969-6ff6-4204-954a-ae927ca88b00-kube-api-access-hdb5b\") pod \"maas-controller-6fbcf7897d-ccsgr\" (UID: \"345ac969-6ff6-4204-954a-ae927ca88b00\") " pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:04.161550 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:04.161462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/345ac969-6ff6-4204-954a-ae927ca88b00-webhook-cert\") pod \"maas-controller-6fbcf7897d-ccsgr\" (UID: \"345ac969-6ff6-4204-954a-ae927ca88b00\") " pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:04.161550 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:04.161504    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdb5b\" (UniqueName: \"kubernetes.io/projected/345ac969-6ff6-4204-954a-ae927ca88b00-kube-api-access-hdb5b\") pod \"maas-controller-6fbcf7897d-ccsgr\" (UID: \"345ac969-6ff6-4204-954a-ae927ca88b00\") " pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:04.164183 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:04.164159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/345ac969-6ff6-4204-954a-ae927ca88b00-webhook-cert\") pod \"maas-controller-6fbcf7897d-ccsgr\" (UID: \"345ac969-6ff6-4204-954a-ae927ca88b00\") " pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:04.173872 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:04.173847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdb5b\" (UniqueName: \"kubernetes.io/projected/345ac969-6ff6-4204-954a-ae927ca88b00-kube-api-access-hdb5b\") pod \"maas-controller-6fbcf7897d-ccsgr\" (UID: \"345ac969-6ff6-4204-954a-ae927ca88b00\") " pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:04.218718 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:04.218691    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:04.352878 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:04.352852    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-ccsgr"]
Jul 27 21:47:04.355601 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:47:04.355570    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345ac969_6ff6_4204_954a_ae927ca88b00.slice/crio-b195b2d780e5aea65ed069db8bbc8642fab68adb62fde4905d4a90593fe76db2 WatchSource:0}: Error finding container b195b2d780e5aea65ed069db8bbc8642fab68adb62fde4905d4a90593fe76db2: Status 404 returned error can't find the container with id b195b2d780e5aea65ed069db8bbc8642fab68adb62fde4905d4a90593fe76db2
Jul 27 21:47:05.322034 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:05.321998    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr" event={"ID":"345ac969-6ff6-4204-954a-ae927ca88b00","Type":"ContainerStarted","Data":"7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87"}
Jul 27 21:47:05.322430 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:05.322061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr" event={"ID":"345ac969-6ff6-4204-954a-ae927ca88b00","Type":"ContainerStarted","Data":"b195b2d780e5aea65ed069db8bbc8642fab68adb62fde4905d4a90593fe76db2"}
Jul 27 21:47:05.322430 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:05.322111    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:05.342433 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:05.342380    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr" podStartSLOduration=1.864940485 podStartE2EDuration="2.342365052s" totalImagesPullingTime="477.424567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:47:05.340655511 +0000 UTC m=+796.236464961" watchObservedRunningTime="2026-07-27 21:47:05.342365052 +0000 UTC m=+796.238174503"
Jul 27 21:47:16.335163 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:16.335086    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:21.006566 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.006531    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-ccsgr"]
Jul 27 21:47:21.006964 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.006764    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr" podUID="345ac969-6ff6-4204-954a-ae927ca88b00" containerName="manager" containerID="cri-o://7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87" gracePeriod=30
Jul 27 21:47:21.261613 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.261555    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:21.311856 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.311827    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdb5b\" (UniqueName: \"kubernetes.io/projected/345ac969-6ff6-4204-954a-ae927ca88b00-kube-api-access-hdb5b\") pod \"345ac969-6ff6-4204-954a-ae927ca88b00\" (UID: \"345ac969-6ff6-4204-954a-ae927ca88b00\") "
Jul 27 21:47:21.312039 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.311928    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/345ac969-6ff6-4204-954a-ae927ca88b00-webhook-cert\") pod \"345ac969-6ff6-4204-954a-ae927ca88b00\" (UID: \"345ac969-6ff6-4204-954a-ae927ca88b00\") "
Jul 27 21:47:21.314053 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.314013    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/345ac969-6ff6-4204-954a-ae927ca88b00-kube-api-access-hdb5b" (OuterVolumeSpecName: "kube-api-access-hdb5b") pod "345ac969-6ff6-4204-954a-ae927ca88b00" (UID: "345ac969-6ff6-4204-954a-ae927ca88b00"). InnerVolumeSpecName "kube-api-access-hdb5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:47:21.314163 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.314074    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/345ac969-6ff6-4204-954a-ae927ca88b00-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "345ac969-6ff6-4204-954a-ae927ca88b00" (UID: "345ac969-6ff6-4204-954a-ae927ca88b00"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:47:21.402813 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.402774    2578 generic.go:356] "Generic (PLEG): container finished" podID="345ac969-6ff6-4204-954a-ae927ca88b00" containerID="7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87" exitCode=0
Jul 27 21:47:21.402977 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.402823    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr"
Jul 27 21:47:21.402977 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.402858    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr" event={"ID":"345ac969-6ff6-4204-954a-ae927ca88b00","Type":"ContainerDied","Data":"7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87"}
Jul 27 21:47:21.402977 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.402916    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-ccsgr" event={"ID":"345ac969-6ff6-4204-954a-ae927ca88b00","Type":"ContainerDied","Data":"b195b2d780e5aea65ed069db8bbc8642fab68adb62fde4905d4a90593fe76db2"}
Jul 27 21:47:21.402977 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.402939    2578 scope.go:117] "RemoveContainer" containerID="7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87"
Jul 27 21:47:21.412485 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.412462    2578 scope.go:117] "RemoveContainer" containerID="7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87"
Jul 27 21:47:21.412795 ip-10-0-132-161 kubenswrapper[2578]: E0727 21:47:21.412771    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87\": container with ID starting with 7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87 not found: ID does not exist" containerID="7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87"
Jul 27 21:47:21.412893 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.412813    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87"} err="failed to get container status \"7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87\": rpc error: code = NotFound desc = could not find container \"7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87\": container with ID starting with 7c9f6eb70521c95211b7513d3346b1b867d857598a7aa33b497c0387d2386c87 not found: ID does not exist"
Jul 27 21:47:21.412893 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.412782    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdb5b\" (UniqueName: \"kubernetes.io/projected/345ac969-6ff6-4204-954a-ae927ca88b00-kube-api-access-hdb5b\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:47:21.412893 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.412849    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/345ac969-6ff6-4204-954a-ae927ca88b00-webhook-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 21:47:21.426308 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.426285    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-ccsgr"]
Jul 27 21:47:21.429439 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.429418    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-ccsgr"]
Jul 27 21:47:21.604928 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:21.604853    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="345ac969-6ff6-4204-954a-ae927ca88b00" path="/var/lib/kubelet/pods/345ac969-6ff6-4204-954a-ae927ca88b00/volumes"
Jul 27 21:47:22.242399 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.242362    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-j27pr"]
Jul 27 21:47:22.242804 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.242686    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="345ac969-6ff6-4204-954a-ae927ca88b00" containerName="manager"
Jul 27 21:47:22.242804 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.242697    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="345ac969-6ff6-4204-954a-ae927ca88b00" containerName="manager"
Jul 27 21:47:22.242804 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.242778    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="345ac969-6ff6-4204-954a-ae927ca88b00" containerName="manager"
Jul 27 21:47:22.246994 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.246973    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:47:22.250604 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.250574    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 21:47:22.250604 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.250576    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wbhc9\""
Jul 27 21:47:22.250795 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.250613    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 21:47:22.255941 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.255920    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-j27pr"]
Jul 27 21:47:22.321397 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.321373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxq64\" (UniqueName: \"kubernetes.io/projected/0460a571-45f1-4f8a-98a0-44744263005f-kube-api-access-jxq64\") pod \"maas-controller-6fbcf7897d-j27pr\" (UID: \"0460a571-45f1-4f8a-98a0-44744263005f\") " pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:47:22.321523 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.321406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0460a571-45f1-4f8a-98a0-44744263005f-webhook-cert\") pod \"maas-controller-6fbcf7897d-j27pr\" (UID: \"0460a571-45f1-4f8a-98a0-44744263005f\") " pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:47:22.421777 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.421746    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxq64\" (UniqueName: \"kubernetes.io/projected/0460a571-45f1-4f8a-98a0-44744263005f-kube-api-access-jxq64\") pod \"maas-controller-6fbcf7897d-j27pr\" (UID: \"0460a571-45f1-4f8a-98a0-44744263005f\") " pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:47:22.421931 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.421788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0460a571-45f1-4f8a-98a0-44744263005f-webhook-cert\") pod \"maas-controller-6fbcf7897d-j27pr\" (UID: \"0460a571-45f1-4f8a-98a0-44744263005f\") " pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:47:22.424328 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.424310    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0460a571-45f1-4f8a-98a0-44744263005f-webhook-cert\") pod \"maas-controller-6fbcf7897d-j27pr\" (UID: \"0460a571-45f1-4f8a-98a0-44744263005f\") " pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:47:22.429945 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.429920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxq64\" (UniqueName: \"kubernetes.io/projected/0460a571-45f1-4f8a-98a0-44744263005f-kube-api-access-jxq64\") pod \"maas-controller-6fbcf7897d-j27pr\" (UID: \"0460a571-45f1-4f8a-98a0-44744263005f\") " pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:47:22.557572 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.557485    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:47:22.689458 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:22.689435    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fbcf7897d-j27pr"]
Jul 27 21:47:22.691401 ip-10-0-132-161 kubenswrapper[2578]: W0727 21:47:22.691369    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0460a571_45f1_4f8a_98a0_44744263005f.slice/crio-aa45765198bc8e5a1a94152142d1488ac0c3c5ae6fa7c13190d03571b1cad8e3 WatchSource:0}: Error finding container aa45765198bc8e5a1a94152142d1488ac0c3c5ae6fa7c13190d03571b1cad8e3: Status 404 returned error can't find the container with id aa45765198bc8e5a1a94152142d1488ac0c3c5ae6fa7c13190d03571b1cad8e3
Jul 27 21:47:23.414720 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:23.414690    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-j27pr" event={"ID":"0460a571-45f1-4f8a-98a0-44744263005f","Type":"ContainerStarted","Data":"c6ba27718d077168727b83f78bdd69e6b17427624310bade3a32e6e0347c75ad"}
Jul 27 21:47:23.415109 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:23.414794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fbcf7897d-j27pr" event={"ID":"0460a571-45f1-4f8a-98a0-44744263005f","Type":"ContainerStarted","Data":"aa45765198bc8e5a1a94152142d1488ac0c3c5ae6fa7c13190d03571b1cad8e3"}
Jul 27 21:47:23.415109 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:23.414817    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:47:23.433962 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:23.433914    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fbcf7897d-j27pr" podStartSLOduration=1.109204917 podStartE2EDuration="1.433898606s" totalImagesPullingTime="324.693689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:47:23.431447651 +0000 UTC m=+814.327257102" watchObservedRunningTime="2026-07-27 21:47:23.433898606 +0000 UTC m=+814.329708056"
Jul 27 21:47:34.422100 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:47:34.422067    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fbcf7897d-j27pr"
Jul 27 21:48:49.509306 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:48:49.509237    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:48:49.509806 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:48:49.509341    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:53:49.539564 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:53:49.539534    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:53:49.542258 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:53:49.542236    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:58:49.566557 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:58:49.566528    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 21:58:49.570532 ip-10-0-132-161 kubenswrapper[2578]: I0727 21:58:49.570509    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:00:00.150447 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:00.150414    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8"]
Jul 27 22:00:00.153641 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:00.153624    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8"
Jul 27 22:00:00.156429 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:00.156409    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxwpp\""
Jul 27 22:00:00.163062 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:00.163046    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8"
Jul 27 22:00:00.178126 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:00.178095    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8"]
Jul 27 22:00:00.301166 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:00.301138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8"]
Jul 27 22:00:00.303635 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:00:00.303603    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1ef333_8aac_4d56_bf42_68e4bde1d487.slice/crio-a4bd5547f6c23358dc255d075f6331a181c161d13d45e41919b4fe9c9ee65b69 WatchSource:0}: Error finding container a4bd5547f6c23358dc255d075f6331a181c161d13d45e41919b4fe9c9ee65b69: Status 404 returned error can't find the container with id a4bd5547f6c23358dc255d075f6331a181c161d13d45e41919b4fe9c9ee65b69
Jul 27 22:00:00.922044 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:00.922010    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8" event={"ID":"cd1ef333-8aac-4d56-bf42-68e4bde1d487","Type":"ContainerStarted","Data":"1f2dc36ba6bbcebd83b2070600a4b9bc96346211abeb608fcb9fc98c297604e1"}
Jul 27 22:00:00.922224 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:00.922073    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8" event={"ID":"cd1ef333-8aac-4d56-bf42-68e4bde1d487","Type":"ContainerStarted","Data":"a4bd5547f6c23358dc255d075f6331a181c161d13d45e41919b4fe9c9ee65b69"}
Jul 27 22:00:00.938442 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:00.938399    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8" podStartSLOduration=0.938384873 podStartE2EDuration="938.384873ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:00:00.93766259 +0000 UTC m=+1571.833472039" watchObservedRunningTime="2026-07-27 22:00:00.938384873 +0000 UTC m=+1571.834194369"
Jul 27 22:00:01.927706 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:01.927670    2578 generic.go:356] "Generic (PLEG): container finished" podID="cd1ef333-8aac-4d56-bf42-68e4bde1d487" containerID="1f2dc36ba6bbcebd83b2070600a4b9bc96346211abeb608fcb9fc98c297604e1" exitCode=0
Jul 27 22:00:01.928159 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:01.927758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8" event={"ID":"cd1ef333-8aac-4d56-bf42-68e4bde1d487","Type":"ContainerDied","Data":"1f2dc36ba6bbcebd83b2070600a4b9bc96346211abeb608fcb9fc98c297604e1"}
Jul 27 22:00:03.066100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:03.066078    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8"
Jul 27 22:00:03.938127 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:03.938095    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8"
Jul 27 22:00:03.938299 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:03.938148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753160-l92k8" event={"ID":"cd1ef333-8aac-4d56-bf42-68e4bde1d487","Type":"ContainerDied","Data":"a4bd5547f6c23358dc255d075f6331a181c161d13d45e41919b4fe9c9ee65b69"}
Jul 27 22:00:03.938299 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:03.938206    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4bd5547f6c23358dc255d075f6331a181c161d13d45e41919b4fe9c9ee65b69"
Jul 27 22:00:37.725752 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.725707    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"]
Jul 27 22:00:37.726183 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.726071    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd1ef333-8aac-4d56-bf42-68e4bde1d487" containerName="cleanup"
Jul 27 22:00:37.726183 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.726083    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd1ef333-8aac-4d56-bf42-68e4bde1d487" containerName="cleanup"
Jul 27 22:00:37.726183 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.726132    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd1ef333-8aac-4d56-bf42-68e4bde1d487" containerName="cleanup"
Jul 27 22:00:37.729115 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.729097    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:37.732126 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.732103    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b8dde9de-serving-cert\""
Jul 27 22:00:37.747278 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.747249    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"]
Jul 27 22:00:37.811744 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.811702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggzcn\" (UniqueName: \"kubernetes.io/projected/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-kube-api-access-ggzcn\") pod \"maas-api-e2e-ait-b8dde9de-766f78968c-sxt78\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:37.811892 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.811830    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-maas-api-tls\") pod \"maas-api-e2e-ait-b8dde9de-766f78968c-sxt78\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:37.811964 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.811936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-openshift-service-ca\") pod \"maas-api-e2e-ait-b8dde9de-766f78968c-sxt78\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:37.912981 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.912949    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-openshift-service-ca\") pod \"maas-api-e2e-ait-b8dde9de-766f78968c-sxt78\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:37.913137 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.912988    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggzcn\" (UniqueName: \"kubernetes.io/projected/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-kube-api-access-ggzcn\") pod \"maas-api-e2e-ait-b8dde9de-766f78968c-sxt78\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:37.913137 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.913024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-maas-api-tls\") pod \"maas-api-e2e-ait-b8dde9de-766f78968c-sxt78\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:37.913596 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.913574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-openshift-service-ca\") pod \"maas-api-e2e-ait-b8dde9de-766f78968c-sxt78\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:37.915430 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.915411    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-maas-api-tls\") pod \"maas-api-e2e-ait-b8dde9de-766f78968c-sxt78\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:37.930465 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:37.930444    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggzcn\" (UniqueName: \"kubernetes.io/projected/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-kube-api-access-ggzcn\") pod \"maas-api-e2e-ait-b8dde9de-766f78968c-sxt78\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:38.039197 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.039149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:38.181633 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.181610    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"]
Jul 27 22:00:38.183268 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:00:38.183242    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode461d7ae_bf1f_45f1_bc14_fd337a03dc5d.slice/crio-125dc5179b4610eb839259373d448d33151200bf8fb1d956cfc8aff05dfe2e97 WatchSource:0}: Error finding container 125dc5179b4610eb839259373d448d33151200bf8fb1d956cfc8aff05dfe2e97: Status 404 returned error can't find the container with id 125dc5179b4610eb839259373d448d33151200bf8fb1d956cfc8aff05dfe2e97
Jul 27 22:00:38.184679 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.184656    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 22:00:38.744375 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.744340    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"]
Jul 27 22:00:38.748913 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.748891    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:00:38.753629 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.753611    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b8dde9de-dockercfg-kgqv4\""
Jul 27 22:00:38.753976 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.753958    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b8dde9de\""
Jul 27 22:00:38.760256 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.760237    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"]
Jul 27 22:00:38.821768 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.821720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv5g4\" (UniqueName: \"kubernetes.io/projected/7f3ea1a8-8473-46cd-b604-00f27d751fa9-kube-api-access-hv5g4\") pod \"payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s\" (UID: \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:00:38.821896 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.821816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f3ea1a8-8473-46cd-b604-00f27d751fa9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s\" (UID: \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:00:38.922183 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.922151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv5g4\" (UniqueName: \"kubernetes.io/projected/7f3ea1a8-8473-46cd-b604-00f27d751fa9-kube-api-access-hv5g4\") pod \"payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s\" (UID: \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:00:38.922315 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.922192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f3ea1a8-8473-46cd-b604-00f27d751fa9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s\" (UID: \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:00:38.922843 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.922827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f3ea1a8-8473-46cd-b604-00f27d751fa9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s\" (UID: \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:00:38.934341 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:38.934310    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv5g4\" (UniqueName: \"kubernetes.io/projected/7f3ea1a8-8473-46cd-b604-00f27d751fa9-kube-api-access-hv5g4\") pod \"payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s\" (UID: \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:00:39.059622 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:39.059570    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:00:39.106692 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:39.106650    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78" event={"ID":"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d","Type":"ContainerStarted","Data":"125dc5179b4610eb839259373d448d33151200bf8fb1d956cfc8aff05dfe2e97"}
Jul 27 22:00:39.206017 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:39.205871    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"]
Jul 27 22:00:39.208718 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:00:39.208691    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3ea1a8_8473_46cd_b604_00f27d751fa9.slice/crio-f28579bc69e6105223ecf63cb44022b2d6c67fc982bf2e0b5cf51b25186a2c2a WatchSource:0}: Error finding container f28579bc69e6105223ecf63cb44022b2d6c67fc982bf2e0b5cf51b25186a2c2a: Status 404 returned error can't find the container with id f28579bc69e6105223ecf63cb44022b2d6c67fc982bf2e0b5cf51b25186a2c2a
Jul 27 22:00:40.113251 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:40.113218    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s" event={"ID":"7f3ea1a8-8473-46cd-b604-00f27d751fa9","Type":"ContainerStarted","Data":"fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1"}
Jul 27 22:00:40.113617 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:40.113267    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s" event={"ID":"7f3ea1a8-8473-46cd-b604-00f27d751fa9","Type":"ContainerStarted","Data":"f28579bc69e6105223ecf63cb44022b2d6c67fc982bf2e0b5cf51b25186a2c2a"}
Jul 27 22:00:40.113617 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:40.113302    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:00:40.134589 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:40.134549    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s" podStartSLOduration=2.134535671 podStartE2EDuration="2.134535671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:00:40.133234408 +0000 UTC m=+1611.029043860" watchObservedRunningTime="2026-07-27 22:00:40.134535671 +0000 UTC m=+1611.030345121"
Jul 27 22:00:41.119970 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:41.119926    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78" event={"ID":"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d","Type":"ContainerStarted","Data":"a0dfbf64cd4212ed74fa27775da4e70116a384c9c87474ac006b8b085ae29d05"}
Jul 27 22:00:41.120435 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:41.120113    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:41.146814 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:41.146763    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78" podStartSLOduration=2.021761471 podStartE2EDuration="4.146745335s" totalImagesPullingTime="2.124983864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:00:41.14440823 +0000 UTC m=+1612.040217681" watchObservedRunningTime="2026-07-27 22:00:41.146745335 +0000 UTC m=+1612.042554780"
Jul 27 22:00:47.130719 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:47.130694    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:00:51.122484 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:00:51.122446    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:01:08.437813 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:08.437776    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"]
Jul 27 22:01:08.441304 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:08.441288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"
Jul 27 22:01:08.445558 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:08.445542    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxwpp\""
Jul 27 22:01:08.457771 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:08.456458    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"]
Jul 27 22:01:08.542981 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:08.542956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/243ac255-7a25-4810-877c-b22661e8ff38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft\" (UID: \"243ac255-7a25-4810-877c-b22661e8ff38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"
Jul 27 22:01:08.644053 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:08.644023    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/243ac255-7a25-4810-877c-b22661e8ff38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft\" (UID: \"243ac255-7a25-4810-877c-b22661e8ff38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"
Jul 27 22:01:08.644568 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:08.644552    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/243ac255-7a25-4810-877c-b22661e8ff38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft\" (UID: \"243ac255-7a25-4810-877c-b22661e8ff38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"
Jul 27 22:01:08.750925 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:08.750877    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"
Jul 27 22:01:08.894718 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:08.894687    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"]
Jul 27 22:01:08.896564 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:01:08.896533    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243ac255_7a25_4810_877c_b22661e8ff38.slice/crio-6a82338e48754076cbb226f5ab02f4b4c9a5fbbf445698c1f9757085d3e5353c WatchSource:0}: Error finding container 6a82338e48754076cbb226f5ab02f4b4c9a5fbbf445698c1f9757085d3e5353c: Status 404 returned error can't find the container with id 6a82338e48754076cbb226f5ab02f4b4c9a5fbbf445698c1f9757085d3e5353c
Jul 27 22:01:09.254776 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:09.254748    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft" event={"ID":"243ac255-7a25-4810-877c-b22661e8ff38","Type":"ContainerStarted","Data":"f0ba3467cf339adbc977aaf33d93d29609a9e116bff7228d5a76c7c9c604b3db"}
Jul 27 22:01:09.254891 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:09.254799    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft" event={"ID":"243ac255-7a25-4810-877c-b22661e8ff38","Type":"ContainerStarted","Data":"6a82338e48754076cbb226f5ab02f4b4c9a5fbbf445698c1f9757085d3e5353c"}
Jul 27 22:01:09.277741 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:09.277695    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft" podStartSLOduration=1.277675211 podStartE2EDuration="1.277675211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:09.277429132 +0000 UTC m=+1640.173238584" watchObservedRunningTime="2026-07-27 22:01:09.277675211 +0000 UTC m=+1640.173484640"
Jul 27 22:01:10.261477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:10.261446    2578 generic.go:356] "Generic (PLEG): container finished" podID="243ac255-7a25-4810-877c-b22661e8ff38" containerID="f0ba3467cf339adbc977aaf33d93d29609a9e116bff7228d5a76c7c9c604b3db" exitCode=0
Jul 27 22:01:10.261840 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:10.261509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft" event={"ID":"243ac255-7a25-4810-877c-b22661e8ff38","Type":"ContainerDied","Data":"f0ba3467cf339adbc977aaf33d93d29609a9e116bff7228d5a76c7c9c604b3db"}
Jul 27 22:01:11.398582 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:11.398557    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"
Jul 27 22:01:11.464192 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:11.464165    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/243ac255-7a25-4810-877c-b22661e8ff38-maas-api-service-ca\") pod \"243ac255-7a25-4810-877c-b22661e8ff38\" (UID: \"243ac255-7a25-4810-877c-b22661e8ff38\") "
Jul 27 22:01:11.464475 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:11.464454    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/243ac255-7a25-4810-877c-b22661e8ff38-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "243ac255-7a25-4810-877c-b22661e8ff38" (UID: "243ac255-7a25-4810-877c-b22661e8ff38"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:01:11.565100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:11.565040    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/243ac255-7a25-4810-877c-b22661e8ff38-maas-api-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:01:12.272025 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:12.271997    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"
Jul 27 22:01:12.272025 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:12.272009    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft" event={"ID":"243ac255-7a25-4810-877c-b22661e8ff38","Type":"ContainerDied","Data":"6a82338e48754076cbb226f5ab02f4b4c9a5fbbf445698c1f9757085d3e5353c"}
Jul 27 22:01:12.272220 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:12.272057    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a82338e48754076cbb226f5ab02f4b4c9a5fbbf445698c1f9757085d3e5353c"
Jul 27 22:01:19.042100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.042069    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"]
Jul 27 22:01:19.042524 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.042293    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78" podUID="e461d7ae-bf1f-45f1-bc14-fd337a03dc5d" containerName="maas-api" containerID="cri-o://a0dfbf64cd4212ed74fa27775da4e70116a384c9c87474ac006b8b085ae29d05" gracePeriod=30
Jul 27 22:01:19.227554 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.227339    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"]
Jul 27 22:01:19.227711 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.227592    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s" podUID="7f3ea1a8-8473-46cd-b604-00f27d751fa9" containerName="payload-pre-processing" containerID="cri-o://fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1" gracePeriod=30
Jul 27 22:01:19.311251 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.311175    2578 generic.go:356] "Generic (PLEG): container finished" podID="e461d7ae-bf1f-45f1-bc14-fd337a03dc5d" containerID="a0dfbf64cd4212ed74fa27775da4e70116a384c9c87474ac006b8b085ae29d05" exitCode=0
Jul 27 22:01:19.311251 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.311215    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78" event={"ID":"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d","Type":"ContainerDied","Data":"a0dfbf64cd4212ed74fa27775da4e70116a384c9c87474ac006b8b085ae29d05"}
Jul 27 22:01:19.353245 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.353224    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:01:19.423456 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.423408    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-maas-api-tls\") pod \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") "
Jul 27 22:01:19.423644 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.423486    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-openshift-service-ca\") pod \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") "
Jul 27 22:01:19.423644 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.423530    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggzcn\" (UniqueName: \"kubernetes.io/projected/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-kube-api-access-ggzcn\") pod \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\" (UID: \"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d\") "
Jul 27 22:01:19.423997 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.423956    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e461d7ae-bf1f-45f1-bc14-fd337a03dc5d" (UID: "e461d7ae-bf1f-45f1-bc14-fd337a03dc5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:01:19.426124 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.426098    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-kube-api-access-ggzcn" (OuterVolumeSpecName: "kube-api-access-ggzcn") pod "e461d7ae-bf1f-45f1-bc14-fd337a03dc5d" (UID: "e461d7ae-bf1f-45f1-bc14-fd337a03dc5d"). InnerVolumeSpecName "kube-api-access-ggzcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:01:19.426327 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.426305    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e461d7ae-bf1f-45f1-bc14-fd337a03dc5d" (UID: "e461d7ae-bf1f-45f1-bc14-fd337a03dc5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:01:19.478447 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.478424    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:01:19.525004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.524982    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f3ea1a8-8473-46cd-b604-00f27d751fa9-plugins-config-volume\") pod \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\" (UID: \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\") "
Jul 27 22:01:19.525102 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.525014    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv5g4\" (UniqueName: \"kubernetes.io/projected/7f3ea1a8-8473-46cd-b604-00f27d751fa9-kube-api-access-hv5g4\") pod \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\" (UID: \"7f3ea1a8-8473-46cd-b604-00f27d751fa9\") "
Jul 27 22:01:19.525166 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.525155    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:01:19.525202 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.525169    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggzcn\" (UniqueName: \"kubernetes.io/projected/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-kube-api-access-ggzcn\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:01:19.525202 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.525179    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:01:19.525375 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.525353    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f3ea1a8-8473-46cd-b604-00f27d751fa9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7f3ea1a8-8473-46cd-b604-00f27d751fa9" (UID: "7f3ea1a8-8473-46cd-b604-00f27d751fa9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:01:19.527269 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.527252    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f3ea1a8-8473-46cd-b604-00f27d751fa9-kube-api-access-hv5g4" (OuterVolumeSpecName: "kube-api-access-hv5g4") pod "7f3ea1a8-8473-46cd-b604-00f27d751fa9" (UID: "7f3ea1a8-8473-46cd-b604-00f27d751fa9"). InnerVolumeSpecName "kube-api-access-hv5g4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:01:19.570720 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.570663    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"]
Jul 27 22:01:19.578296 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.578265    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8dde9de-ghdft"]
Jul 27 22:01:19.610743 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.610696    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="243ac255-7a25-4810-877c-b22661e8ff38" path="/var/lib/kubelet/pods/243ac255-7a25-4810-877c-b22661e8ff38/volumes"
Jul 27 22:01:19.625746 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.625719    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f3ea1a8-8473-46cd-b604-00f27d751fa9-plugins-config-volume\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:01:19.625828 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:19.625761    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv5g4\" (UniqueName: \"kubernetes.io/projected/7f3ea1a8-8473-46cd-b604-00f27d751fa9-kube-api-access-hv5g4\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:01:20.317268 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.317196    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"
Jul 27 22:01:20.317268 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.317217    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78" event={"ID":"e461d7ae-bf1f-45f1-bc14-fd337a03dc5d","Type":"ContainerDied","Data":"125dc5179b4610eb839259373d448d33151200bf8fb1d956cfc8aff05dfe2e97"}
Jul 27 22:01:20.317802 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.317291    2578 scope.go:117] "RemoveContainer" containerID="a0dfbf64cd4212ed74fa27775da4e70116a384c9c87474ac006b8b085ae29d05"
Jul 27 22:01:20.318551 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.318520    2578 generic.go:356] "Generic (PLEG): container finished" podID="7f3ea1a8-8473-46cd-b604-00f27d751fa9" containerID="fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1" exitCode=0
Jul 27 22:01:20.318621 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.318565    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s" event={"ID":"7f3ea1a8-8473-46cd-b604-00f27d751fa9","Type":"ContainerDied","Data":"fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1"}
Jul 27 22:01:20.318621 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.318602    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s" event={"ID":"7f3ea1a8-8473-46cd-b604-00f27d751fa9","Type":"ContainerDied","Data":"f28579bc69e6105223ecf63cb44022b2d6c67fc982bf2e0b5cf51b25186a2c2a"}
Jul 27 22:01:20.318752 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.318622    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"
Jul 27 22:01:20.325944 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.325925    2578 scope.go:117] "RemoveContainer" containerID="fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1"
Jul 27 22:01:20.333398 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.333380    2578 scope.go:117] "RemoveContainer" containerID="fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1"
Jul 27 22:01:20.333641 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:01:20.333624    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1\": container with ID starting with fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1 not found: ID does not exist" containerID="fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1"
Jul 27 22:01:20.333706 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.333653    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1"} err="failed to get container status \"fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1\": rpc error: code = NotFound desc = could not find container \"fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1\": container with ID starting with fe17e307c57a3c78c90106f67c5fb0556272475281452af6d0238f9380e428d1 not found: ID does not exist"
Jul 27 22:01:20.373941 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.373886    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"]
Jul 27 22:01:20.387159 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.387139    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8dde9de-596fb69548-mjq9s"]
Jul 27 22:01:20.417902 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.417879    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"]
Jul 27 22:01:20.425411 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:20.425391    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b8dde9de-766f78968c-sxt78"]
Jul 27 22:01:21.038714 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.038683    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"]
Jul 27 22:01:21.039072 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.039059    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="243ac255-7a25-4810-877c-b22661e8ff38" containerName="revoke-keys"
Jul 27 22:01:21.039119 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.039075    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="243ac255-7a25-4810-877c-b22661e8ff38" containerName="revoke-keys"
Jul 27 22:01:21.039119 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.039090    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f3ea1a8-8473-46cd-b604-00f27d751fa9" containerName="payload-pre-processing"
Jul 27 22:01:21.039119 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.039095    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f3ea1a8-8473-46cd-b604-00f27d751fa9" containerName="payload-pre-processing"
Jul 27 22:01:21.039119 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.039102    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e461d7ae-bf1f-45f1-bc14-fd337a03dc5d" containerName="maas-api"
Jul 27 22:01:21.039119 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.039108    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e461d7ae-bf1f-45f1-bc14-fd337a03dc5d" containerName="maas-api"
Jul 27 22:01:21.039262 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.039162    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="243ac255-7a25-4810-877c-b22661e8ff38" containerName="revoke-keys"
Jul 27 22:01:21.039262 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.039173    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e461d7ae-bf1f-45f1-bc14-fd337a03dc5d" containerName="maas-api"
Jul 27 22:01:21.039262 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.039179    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f3ea1a8-8473-46cd-b604-00f27d751fa9" containerName="payload-pre-processing"
Jul 27 22:01:21.043627 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.043608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.046900 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.046878    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d38e15cc-gw-openshift-default-dockercfg-zglfp\""
Jul 27 22:01:21.070133 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.070105    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"]
Jul 27 22:01:21.139988 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.139951    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-podinfo\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.140145 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.140052    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-envoy\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.140217 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.140178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-data\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.140278 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.140256    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-socket\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.140361 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.140337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-credential-socket\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.140427 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.140377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbqft\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-kube-api-access-qbqft\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.140489 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.140454    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5adced16-de7c-4de3-b8b3-5a767df1a725-istiod-ca-cert\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.140535 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.140490    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-certs\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.140535 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.140517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-token\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.241726 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.241698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-socket\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.241726 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.241726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-credential-socket\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.241932 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.241793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbqft\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-kube-api-access-qbqft\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.241932 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.241842    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5adced16-de7c-4de3-b8b3-5a767df1a725-istiod-ca-cert\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.241932 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.241866    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-certs\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.241932 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.241884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-token\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.241932 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.241913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-podinfo\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.242171 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.241959    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-envoy\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.242171 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.242019    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-data\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.242171 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.242112    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-credential-socket\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.242299 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.242205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-certs\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.242299 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.242247    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-socket\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.242424 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.242404    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-data\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.242552 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.242534    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5adced16-de7c-4de3-b8b3-5a767df1a725-istiod-ca-cert\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.244534 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.244509    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-envoy\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.244690 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.244674    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-podinfo\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.252400 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.252380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-token\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.254635 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.254612    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbqft\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-kube-api-access-qbqft\") pod \"e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") " pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.353903 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.353850    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:21.497015 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.496987    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"]
Jul 27 22:01:21.500311 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:01:21.500286    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5adced16_de7c_4de3_b8b3_5a767df1a725.slice/crio-1e4d537d9ce2e1813c93b7c049141f703af5a9bf44c7b0a81d3f6f4747c87bab WatchSource:0}: Error finding container 1e4d537d9ce2e1813c93b7c049141f703af5a9bf44c7b0a81d3f6f4747c87bab: Status 404 returned error can't find the container with id 1e4d537d9ce2e1813c93b7c049141f703af5a9bf44c7b0a81d3f6f4747c87bab
Jul 27 22:01:21.605505 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.605452    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f3ea1a8-8473-46cd-b604-00f27d751fa9" path="/var/lib/kubelet/pods/7f3ea1a8-8473-46cd-b604-00f27d751fa9/volumes"
Jul 27 22:01:21.605897 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:21.605880    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e461d7ae-bf1f-45f1-bc14-fd337a03dc5d" path="/var/lib/kubelet/pods/e461d7ae-bf1f-45f1-bc14-fd337a03dc5d/volumes"
Jul 27 22:01:22.332577 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:22.332529    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" event={"ID":"5adced16-de7c-4de3-b8b3-5a767df1a725","Type":"ContainerStarted","Data":"1e4d537d9ce2e1813c93b7c049141f703af5a9bf44c7b0a81d3f6f4747c87bab"}
Jul 27 22:01:24.225410 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:24.225375    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:01:24.225709 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:24.225446    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:01:24.225709 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:24.225478    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:01:24.345515 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:24.345476    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" event={"ID":"5adced16-de7c-4de3-b8b3-5a767df1a725","Type":"ContainerStarted","Data":"d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a"}
Jul 27 22:01:24.354297 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:24.354269    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:24.374524 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:24.374466    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" podStartSLOduration=1.6515000899999999 podStartE2EDuration="4.374445323s" totalImagesPullingTime="2.722945233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:24.370867699 +0000 UTC m=+1655.266677143" watchObservedRunningTime="2026-07-27 22:01:24.374445323 +0000 UTC m=+1655.270254776"
Jul 27 22:01:25.359223 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:25.359194    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:26.047277 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.047244    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"]
Jul 27 22:01:26.050826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.050811    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.056204 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.056183    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d38e15cc-serving-cert\""
Jul 27 22:01:26.083446 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.083422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785c9af6-31e9-42a1-9fad-ade815080493-maas-api-tls\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.083547 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.083445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785c9af6-31e9-42a1-9fad-ade815080493-openshift-service-ca\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.083589 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.083539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gct84\" (UniqueName: \"kubernetes.io/projected/785c9af6-31e9-42a1-9fad-ade815080493-kube-api-access-gct84\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.129122 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.129094    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"]
Jul 27 22:01:26.184716 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.184692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gct84\" (UniqueName: \"kubernetes.io/projected/785c9af6-31e9-42a1-9fad-ade815080493-kube-api-access-gct84\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.184863 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.184774    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785c9af6-31e9-42a1-9fad-ade815080493-maas-api-tls\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.184863 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.184793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785c9af6-31e9-42a1-9fad-ade815080493-openshift-service-ca\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.184946 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:01:26.184908    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-serving-cert: secret "maas-api-e2e-migrate-d38e15cc-serving-cert" not found
Jul 27 22:01:26.184997 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:01:26.184986    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/785c9af6-31e9-42a1-9fad-ade815080493-maas-api-tls podName:785c9af6-31e9-42a1-9fad-ade815080493 nodeName:}" failed. No retries permitted until 2026-07-27 22:01:26.684968679 +0000 UTC m=+1657.580778128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/785c9af6-31e9-42a1-9fad-ade815080493-maas-api-tls") pod "maas-api-e2e-migrate-d38e15cc-7958588849-7hh87" (UID: "785c9af6-31e9-42a1-9fad-ade815080493") : secret "maas-api-e2e-migrate-d38e15cc-serving-cert" not found
Jul 27 22:01:26.185317 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.185301    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785c9af6-31e9-42a1-9fad-ade815080493-openshift-service-ca\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.197281 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.197256    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gct84\" (UniqueName: \"kubernetes.io/projected/785c9af6-31e9-42a1-9fad-ade815080493-kube-api-access-gct84\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.361227 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.361162    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:26.362150 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.362131    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:26.689038 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.689007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785c9af6-31e9-42a1-9fad-ade815080493-maas-api-tls\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.691529 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.691505    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785c9af6-31e9-42a1-9fad-ade815080493-maas-api-tls\") pod \"maas-api-e2e-migrate-d38e15cc-7958588849-7hh87\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:26.960512 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:26.960442    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:27.122438 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:27.122417    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"]
Jul 27 22:01:27.125090 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:01:27.125064    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785c9af6_31e9_42a1_9fad_ade815080493.slice/crio-62fcea1291598c0fc306cb881ea96fd584b1714d4331c2d7de86f2a60ac882c3 WatchSource:0}: Error finding container 62fcea1291598c0fc306cb881ea96fd584b1714d4331c2d7de86f2a60ac882c3: Status 404 returned error can't find the container with id 62fcea1291598c0fc306cb881ea96fd584b1714d4331c2d7de86f2a60ac882c3
Jul 27 22:01:27.367721 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:27.367646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87" event={"ID":"785c9af6-31e9-42a1-9fad-ade815080493","Type":"ContainerStarted","Data":"62fcea1291598c0fc306cb881ea96fd584b1714d4331c2d7de86f2a60ac882c3"}
Jul 27 22:01:28.373751 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:28.373699    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87" event={"ID":"785c9af6-31e9-42a1-9fad-ade815080493","Type":"ContainerStarted","Data":"38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256"}
Jul 27 22:01:28.374190 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:28.374015    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:28.527326 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:28.527279    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87" podStartSLOduration=2.063289223 podStartE2EDuration="2.527264576s" totalImagesPullingTime="463.975353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:28.525913523 +0000 UTC m=+1659.421722975" watchObservedRunningTime="2026-07-27 22:01:28.527264576 +0000 UTC m=+1659.423074027"
Jul 27 22:01:34.383686 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:34.383651    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:39.429868 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:39.429836    2578 generic.go:356] "Generic (PLEG): container finished" podID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerID="d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a" exitCode=0
Jul 27 22:01:39.430278 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:39.429910    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" event={"ID":"5adced16-de7c-4de3-b8b3-5a767df1a725","Type":"ContainerDied","Data":"d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a"}
Jul 27 22:01:39.430278 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:39.430240    2578 scope.go:117] "RemoveContainer" containerID="d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a"
Jul 27 22:01:39.431310 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:39.431275    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:01:39.431376 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:39.431353    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:01:39.431416 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:39.431389    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:01:40.436352 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:40.436318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" event={"ID":"5adced16-de7c-4de3-b8b3-5a767df1a725","Type":"ContainerStarted","Data":"04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc"}
Jul 27 22:01:41.354601 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:41.354569    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:41.355799 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:41.355767    2578 patch_prober.go:28] interesting pod/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 27 22:01:41.355999 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:41.355974    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 27 22:01:41.361206 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:41.361178    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:42.355069 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:42.355029    2578 patch_prober.go:28] interesting pod/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 27 22:01:42.355650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:42.355616    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 27 22:01:43.354637 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:43.354605    2578 patch_prober.go:28] interesting pod/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 27 22:01:43.354912 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:43.354884    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 27 22:01:44.355011 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:44.354976    2578 patch_prober.go:28] interesting pod/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 27 22:01:44.355604 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:44.355570    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 27 22:01:45.358133 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:45.358107    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:45.359037 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:45.359020    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:01:57.720041 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:57.720007    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"]
Jul 27 22:01:57.720559 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:57.720251    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87" podUID="785c9af6-31e9-42a1-9fad-ade815080493" containerName="maas-api" containerID="cri-o://38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256" gracePeriod=30
Jul 27 22:01:57.975215 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:57.975150    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:58.061383 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.061355    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gct84\" (UniqueName: \"kubernetes.io/projected/785c9af6-31e9-42a1-9fad-ade815080493-kube-api-access-gct84\") pod \"785c9af6-31e9-42a1-9fad-ade815080493\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") "
Jul 27 22:01:58.061543 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.061398    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785c9af6-31e9-42a1-9fad-ade815080493-maas-api-tls\") pod \"785c9af6-31e9-42a1-9fad-ade815080493\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") "
Jul 27 22:01:58.061543 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.061420    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785c9af6-31e9-42a1-9fad-ade815080493-openshift-service-ca\") pod \"785c9af6-31e9-42a1-9fad-ade815080493\" (UID: \"785c9af6-31e9-42a1-9fad-ade815080493\") "
Jul 27 22:01:58.061817 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.061797    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/785c9af6-31e9-42a1-9fad-ade815080493-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "785c9af6-31e9-42a1-9fad-ade815080493" (UID: "785c9af6-31e9-42a1-9fad-ade815080493"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:01:58.063539 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.063519    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785c9af6-31e9-42a1-9fad-ade815080493-kube-api-access-gct84" (OuterVolumeSpecName: "kube-api-access-gct84") pod "785c9af6-31e9-42a1-9fad-ade815080493" (UID: "785c9af6-31e9-42a1-9fad-ade815080493"). InnerVolumeSpecName "kube-api-access-gct84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:01:58.063601 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.063535    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/785c9af6-31e9-42a1-9fad-ade815080493-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "785c9af6-31e9-42a1-9fad-ade815080493" (UID: "785c9af6-31e9-42a1-9fad-ade815080493"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:01:58.162436 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.162405    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gct84\" (UniqueName: \"kubernetes.io/projected/785c9af6-31e9-42a1-9fad-ade815080493-kube-api-access-gct84\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:01:58.162436 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.162429    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785c9af6-31e9-42a1-9fad-ade815080493-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:01:58.162436 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.162438    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785c9af6-31e9-42a1-9fad-ade815080493-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:01:58.526383 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.526346    2578 generic.go:356] "Generic (PLEG): container finished" podID="785c9af6-31e9-42a1-9fad-ade815080493" containerID="38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256" exitCode=0
Jul 27 22:01:58.526383 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.526385    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"
Jul 27 22:01:58.526587 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.526434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87" event={"ID":"785c9af6-31e9-42a1-9fad-ade815080493","Type":"ContainerDied","Data":"38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256"}
Jul 27 22:01:58.526587 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.526483    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87" event={"ID":"785c9af6-31e9-42a1-9fad-ade815080493","Type":"ContainerDied","Data":"62fcea1291598c0fc306cb881ea96fd584b1714d4331c2d7de86f2a60ac882c3"}
Jul 27 22:01:58.526587 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.526506    2578 scope.go:117] "RemoveContainer" containerID="38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256"
Jul 27 22:01:58.536677 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.536661    2578 scope.go:117] "RemoveContainer" containerID="38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256"
Jul 27 22:01:58.536949 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:01:58.536932    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256\": container with ID starting with 38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256 not found: ID does not exist" containerID="38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256"
Jul 27 22:01:58.536995 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.536958    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256"} err="failed to get container status \"38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256\": rpc error: code = NotFound desc = could not find container \"38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256\": container with ID starting with 38a4932d0eb9984ef3b30ad57f3faaa5af4e6c08ec23ad99f2c94d8e1b843256 not found: ID does not exist"
Jul 27 22:01:58.591464 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.591445    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"]
Jul 27 22:01:58.611660 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.611637    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d38e15cc-7958588849-7hh87"]
Jul 27 22:01:58.670754 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.670418    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"]
Jul 27 22:01:58.671758 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:58.670959    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy" containerID="cri-o://04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc" gracePeriod=30
Jul 27 22:01:59.606301 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.606268    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="785c9af6-31e9-42a1-9fad-ade815080493" path="/var/lib/kubelet/pods/785c9af6-31e9-42a1-9fad-ade815080493/volumes"
Jul 27 22:01:59.637437 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.637409    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"]
Jul 27 22:01:59.637772 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.637756    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="785c9af6-31e9-42a1-9fad-ade815080493" containerName="maas-api"
Jul 27 22:01:59.637772 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.637770    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="785c9af6-31e9-42a1-9fad-ade815080493" containerName="maas-api"
Jul 27 22:01:59.637855 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.637824    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="785c9af6-31e9-42a1-9fad-ade815080493" containerName="maas-api"
Jul 27 22:01:59.642379 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.642363    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.652278 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.652258    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-58e80fe2-openshift-default-dockercfg-lcxg7\""
Jul 27 22:01:59.672137 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-token\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.672258 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672194    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istiod-ca-cert\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.672258 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672227    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-credential-socket\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.672363 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672275    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-data\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.672363 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672325    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-certs\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.672467 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672423    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-socket\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.672509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672467    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-envoy\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.672509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-podinfo\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.672593 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672519    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n72bf\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-kube-api-access-n72bf\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.672907 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.672884    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"]
Jul 27 22:01:59.773546 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.773508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-certs\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.773773 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.773555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-socket\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.773773 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.773585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-envoy\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.773773 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.773606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-podinfo\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.773773 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.773631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n72bf\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-kube-api-access-n72bf\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.773773 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.773704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-token\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.774064 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.773797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istiod-ca-cert\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.774064 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.773829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-credential-socket\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.774064 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.773979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-data\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.774064 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.774015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-socket\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.774269 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.774069    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-certs\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.774495 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.774263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-credential-socket\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.774495 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.774340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-data\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.774665 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.774541    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istiod-ca-cert\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.776374 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.776348    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-envoy\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.776473 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.776387    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-podinfo\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.809827 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.809795    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-token\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.811176 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.811156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n72bf\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-kube-api-access-n72bf\") pod \"e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") " pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:01:59.953206 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:01:59.953178    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:00.171169 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.171144    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"]
Jul 27 22:02:00.172817 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:02:00.172790    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652132ce_5416_4c89_9ca7_ef8a74d15fd2.slice/crio-e943b161942ceddd0527806db06ebffb966ca06c6eaefd60ea58f080feecf661 WatchSource:0}: Error finding container e943b161942ceddd0527806db06ebffb966ca06c6eaefd60ea58f080feecf661: Status 404 returned error can't find the container with id e943b161942ceddd0527806db06ebffb966ca06c6eaefd60ea58f080feecf661
Jul 27 22:02:00.174868 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.174835    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:02:00.174968 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.174908    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:02:00.174968 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.174955    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:02:00.361658 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.361620    2578 patch_prober.go:28] interesting pod/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 27 22:02:00.361840 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.361687    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 22:02:00.541104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.541020    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" event={"ID":"652132ce-5416-4c89-9ca7-ef8a74d15fd2","Type":"ContainerStarted","Data":"bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30"}
Jul 27 22:02:00.541104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.541083    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" event={"ID":"652132ce-5416-4c89-9ca7-ef8a74d15fd2","Type":"ContainerStarted","Data":"e943b161942ceddd0527806db06ebffb966ca06c6eaefd60ea58f080feecf661"}
Jul 27 22:02:00.575075 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.575021    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" podStartSLOduration=1.5750078570000001 podStartE2EDuration="1.575007857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:00.574894906 +0000 UTC m=+1691.470704358" watchObservedRunningTime="2026-07-27 22:02:00.575007857 +0000 UTC m=+1691.470817306"
Jul 27 22:02:00.953907 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:00.953870    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:01.959138 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:01.959108    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:02.312438 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.312354    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"]
Jul 27 22:02:02.315769 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.315750    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:02.326444 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.326425    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-58e80fe2-serving-cert\""
Jul 27 22:02:02.348205 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.348176    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"]
Jul 27 22:02:02.393212 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.393178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d9b571-c89b-4520-b2f2-f0a790f5985b-openshift-service-ca\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:02.393376 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.393221    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99d9b571-c89b-4520-b2f2-f0a790f5985b-maas-api-tls\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:02.393376 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.393297    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5624\" (UniqueName: \"kubernetes.io/projected/99d9b571-c89b-4520-b2f2-f0a790f5985b-kube-api-access-x5624\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:02.494177 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.494150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d9b571-c89b-4520-b2f2-f0a790f5985b-openshift-service-ca\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:02.494340 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.494192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99d9b571-c89b-4520-b2f2-f0a790f5985b-maas-api-tls\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:02.494340 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.494221    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5624\" (UniqueName: \"kubernetes.io/projected/99d9b571-c89b-4520-b2f2-f0a790f5985b-kube-api-access-x5624\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:02.494340 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:02.494314    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-serving-cert: secret "maas-api-e2e-ait-58e80fe2-serving-cert" not found
Jul 27 22:02:02.494440 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:02.494389    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99d9b571-c89b-4520-b2f2-f0a790f5985b-maas-api-tls podName:99d9b571-c89b-4520-b2f2-f0a790f5985b nodeName:}" failed. No retries permitted until 2026-07-27 22:02:02.994367339 +0000 UTC m=+1693.890176781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99d9b571-c89b-4520-b2f2-f0a790f5985b-maas-api-tls") pod "maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z" (UID: "99d9b571-c89b-4520-b2f2-f0a790f5985b") : secret "maas-api-e2e-ait-58e80fe2-serving-cert" not found
Jul 27 22:02:02.494887 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.494869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d9b571-c89b-4520-b2f2-f0a790f5985b-openshift-service-ca\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:02.513246 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.513223    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5624\" (UniqueName: \"kubernetes.io/projected/99d9b571-c89b-4520-b2f2-f0a790f5985b-kube-api-access-x5624\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:02.551671 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.551651    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:02.552583 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.552563    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:02.999536 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:02.999501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99d9b571-c89b-4520-b2f2-f0a790f5985b-maas-api-tls\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:03.001965 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:03.001948    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99d9b571-c89b-4520-b2f2-f0a790f5985b-maas-api-tls\") pod \"maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:03.226589 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:03.226558    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:03.393029 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:03.392996    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"]
Jul 27 22:02:03.397591 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:02:03.397555    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d9b571_c89b_4520_b2f2_f0a790f5985b.slice/crio-e5f5237ad5fdb8b109953654f15f9888b33d4326dd29c45e245634b630995584 WatchSource:0}: Error finding container e5f5237ad5fdb8b109953654f15f9888b33d4326dd29c45e245634b630995584: Status 404 returned error can't find the container with id e5f5237ad5fdb8b109953654f15f9888b33d4326dd29c45e245634b630995584
Jul 27 22:02:03.557473 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:03.557372    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z" event={"ID":"99d9b571-c89b-4520-b2f2-f0a790f5985b","Type":"ContainerStarted","Data":"e5f5237ad5fdb8b109953654f15f9888b33d4326dd29c45e245634b630995584"}
Jul 27 22:02:03.948405 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:03.948377    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:02:04.007222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007192    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-podinfo\") pod \"5adced16-de7c-4de3-b8b3-5a767df1a725\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") "
Jul 27 22:02:04.007676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007237    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-token\") pod \"5adced16-de7c-4de3-b8b3-5a767df1a725\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") "
Jul 27 22:02:04.007676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007278    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5adced16-de7c-4de3-b8b3-5a767df1a725-istiod-ca-cert\") pod \"5adced16-de7c-4de3-b8b3-5a767df1a725\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") "
Jul 27 22:02:04.007676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007309    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-envoy\") pod \"5adced16-de7c-4de3-b8b3-5a767df1a725\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") "
Jul 27 22:02:04.007676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007350    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-data\") pod \"5adced16-de7c-4de3-b8b3-5a767df1a725\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") "
Jul 27 22:02:04.007676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007372    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-certs\") pod \"5adced16-de7c-4de3-b8b3-5a767df1a725\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") "
Jul 27 22:02:04.007676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007407    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-socket\") pod \"5adced16-de7c-4de3-b8b3-5a767df1a725\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") "
Jul 27 22:02:04.007676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007447    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbqft\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-kube-api-access-qbqft\") pod \"5adced16-de7c-4de3-b8b3-5a767df1a725\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") "
Jul 27 22:02:04.007676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007519    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-credential-socket\") pod \"5adced16-de7c-4de3-b8b3-5a767df1a725\" (UID: \"5adced16-de7c-4de3-b8b3-5a767df1a725\") "
Jul 27 22:02:04.008143 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007784    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5adced16-de7c-4de3-b8b3-5a767df1a725" (UID: "5adced16-de7c-4de3-b8b3-5a767df1a725"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:04.008143 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.007822    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5adced16-de7c-4de3-b8b3-5a767df1a725-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5adced16-de7c-4de3-b8b3-5a767df1a725" (UID: "5adced16-de7c-4de3-b8b3-5a767df1a725"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:02:04.008579 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.008542    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5adced16-de7c-4de3-b8b3-5a767df1a725" (UID: "5adced16-de7c-4de3-b8b3-5a767df1a725"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:04.008819 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.008720    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5adced16-de7c-4de3-b8b3-5a767df1a725" (UID: "5adced16-de7c-4de3-b8b3-5a767df1a725"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:04.009484 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.009444    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-data" (OuterVolumeSpecName: "istio-data") pod "5adced16-de7c-4de3-b8b3-5a767df1a725" (UID: "5adced16-de7c-4de3-b8b3-5a767df1a725"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:04.010202 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.010025    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5adced16-de7c-4de3-b8b3-5a767df1a725" (UID: "5adced16-de7c-4de3-b8b3-5a767df1a725"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:02:04.010638 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.010614    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-kube-api-access-qbqft" (OuterVolumeSpecName: "kube-api-access-qbqft") pod "5adced16-de7c-4de3-b8b3-5a767df1a725" (UID: "5adced16-de7c-4de3-b8b3-5a767df1a725"). InnerVolumeSpecName "kube-api-access-qbqft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:04.010827 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.010682    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-token" (OuterVolumeSpecName: "istio-token") pod "5adced16-de7c-4de3-b8b3-5a767df1a725" (UID: "5adced16-de7c-4de3-b8b3-5a767df1a725"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:04.011624 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.010933    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5adced16-de7c-4de3-b8b3-5a767df1a725" (UID: "5adced16-de7c-4de3-b8b3-5a767df1a725"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:04.109212 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.109137    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:04.109212 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.109162    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:04.109212 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.109171    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:04.109212 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.109178    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:04.109212 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.109189    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbqft\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-kube-api-access-qbqft\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:04.109212 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.109199    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adced16-de7c-4de3-b8b3-5a767df1a725-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:04.109212 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.109207    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:04.109212 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.109216    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5adced16-de7c-4de3-b8b3-5a767df1a725-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:04.109553 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.109224    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5adced16-de7c-4de3-b8b3-5a767df1a725-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:04.563586 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.563550    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z" event={"ID":"99d9b571-c89b-4520-b2f2-f0a790f5985b","Type":"ContainerStarted","Data":"264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8"}
Jul 27 22:02:04.563745 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.563641    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:04.565401 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.565375    2578 generic.go:356] "Generic (PLEG): container finished" podID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerID="04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc" exitCode=0
Jul 27 22:02:04.565553 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.565403    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" event={"ID":"5adced16-de7c-4de3-b8b3-5a767df1a725","Type":"ContainerDied","Data":"04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc"}
Jul 27 22:02:04.565553 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.565434    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"
Jul 27 22:02:04.565553 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.565446    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz" event={"ID":"5adced16-de7c-4de3-b8b3-5a767df1a725","Type":"ContainerDied","Data":"1e4d537d9ce2e1813c93b7c049141f703af5a9bf44c7b0a81d3f6f4747c87bab"}
Jul 27 22:02:04.565553 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.565470    2578 scope.go:117] "RemoveContainer" containerID="04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc"
Jul 27 22:02:04.575434 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.575417    2578 scope.go:117] "RemoveContainer" containerID="d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a"
Jul 27 22:02:04.583614 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.583597    2578 scope.go:117] "RemoveContainer" containerID="04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc"
Jul 27 22:02:04.583928 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:04.583908    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc\": container with ID starting with 04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc not found: ID does not exist" containerID="04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc"
Jul 27 22:02:04.583998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.583935    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc"} err="failed to get container status \"04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc\": rpc error: code = NotFound desc = could not find container \"04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc\": container with ID starting with 04b8413343e13256e869322b1de431c443e60cc29c6be0b54913428a9d025dbc not found: ID does not exist"
Jul 27 22:02:04.583998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.583953    2578 scope.go:117] "RemoveContainer" containerID="d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a"
Jul 27 22:02:04.584234 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:04.584217    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a\": container with ID starting with d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a not found: ID does not exist" containerID="d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a"
Jul 27 22:02:04.584282 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.584240    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a"} err="failed to get container status \"d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a\": rpc error: code = NotFound desc = could not find container \"d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a\": container with ID starting with d8ae4f5b00f3b5d6851fc3d59f8ecd229ed96c9ab651ce198ba363878201cf4a not found: ID does not exist"
Jul 27 22:02:04.625405 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.625355    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z" podStartSLOduration=2.208502551 podStartE2EDuration="2.625342579s" totalImagesPullingTime="416.840028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:02:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:04.623696155 +0000 UTC m=+1695.519505606" watchObservedRunningTime="2026-07-27 22:02:04.625342579 +0000 UTC m=+1695.521152029"
Jul 27 22:02:04.658036 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.658007    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"]
Jul 27 22:02:04.661751 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:04.661715    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d38e15cc-gw-openshift-default-6d68d6b66d-fwltz"]
Jul 27 22:02:05.605513 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:05.605482    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" path="/var/lib/kubelet/pods/5adced16-de7c-4de3-b8b3-5a767df1a725/volumes"
Jul 27 22:02:10.578524 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:10.578442    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:17.552945 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:17.552912    2578 patch_prober.go:28] interesting pod/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 27 22:02:17.553337 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:17.552961    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 27 22:02:17.633213 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:17.633181    2578 generic.go:356] "Generic (PLEG): container finished" podID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerID="bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30" exitCode=0
Jul 27 22:02:17.633576 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:17.633259    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" event={"ID":"652132ce-5416-4c89-9ca7-ef8a74d15fd2","Type":"ContainerDied","Data":"bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30"}
Jul 27 22:02:17.633654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:17.633635    2578 scope.go:117] "RemoveContainer" containerID="bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30"
Jul 27 22:02:17.634611 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:17.634582    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:02:17.634683 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:17.634665    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:02:17.634770 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:17.634755    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:02:18.640495 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:18.640463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" event={"ID":"652132ce-5416-4c89-9ca7-ef8a74d15fd2","Type":"ContainerStarted","Data":"35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251"}
Jul 27 22:02:18.953987 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:18.953951    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:18.954192 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:18.954081    2578 patch_prober.go:28] interesting pod/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 27 22:02:18.954192 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:18.954135    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 27 22:02:19.954676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:19.954635    2578 patch_prober.go:28] interesting pod/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 27 22:02:19.955186 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:19.954697    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 27 22:02:20.954879 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:20.954401    2578 patch_prober.go:28] interesting pod/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 27 22:02:20.954879 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:20.954461    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 27 22:02:21.954391 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:21.954350    2578 patch_prober.go:28] interesting pod/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 27 22:02:21.954593 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:21.954419    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 27 22:02:22.959777 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:22.957929    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:22.959777 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:22.959649    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:22.965530 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:22.965503    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:33.669014 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:33.668980    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"]
Jul 27 22:02:33.669539 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:33.669201    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z" podUID="99d9b571-c89b-4520-b2f2-f0a790f5985b" containerName="maas-api" containerID="cri-o://264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8" gracePeriod=30
Jul 27 22:02:33.923221 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:33.923154    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:34.077109 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.077076    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5624\" (UniqueName: \"kubernetes.io/projected/99d9b571-c89b-4520-b2f2-f0a790f5985b-kube-api-access-x5624\") pod \"99d9b571-c89b-4520-b2f2-f0a790f5985b\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") "
Jul 27 22:02:34.077291 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.077122    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99d9b571-c89b-4520-b2f2-f0a790f5985b-maas-api-tls\") pod \"99d9b571-c89b-4520-b2f2-f0a790f5985b\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") "
Jul 27 22:02:34.077291 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.077147    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d9b571-c89b-4520-b2f2-f0a790f5985b-openshift-service-ca\") pod \"99d9b571-c89b-4520-b2f2-f0a790f5985b\" (UID: \"99d9b571-c89b-4520-b2f2-f0a790f5985b\") "
Jul 27 22:02:34.077510 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.077487    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d9b571-c89b-4520-b2f2-f0a790f5985b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99d9b571-c89b-4520-b2f2-f0a790f5985b" (UID: "99d9b571-c89b-4520-b2f2-f0a790f5985b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:02:34.079395 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.079372    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d9b571-c89b-4520-b2f2-f0a790f5985b-kube-api-access-x5624" (OuterVolumeSpecName: "kube-api-access-x5624") pod "99d9b571-c89b-4520-b2f2-f0a790f5985b" (UID: "99d9b571-c89b-4520-b2f2-f0a790f5985b"). InnerVolumeSpecName "kube-api-access-x5624". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:34.079395 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.079386    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99d9b571-c89b-4520-b2f2-f0a790f5985b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99d9b571-c89b-4520-b2f2-f0a790f5985b" (UID: "99d9b571-c89b-4520-b2f2-f0a790f5985b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:02:34.178698 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.178605    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5624\" (UniqueName: \"kubernetes.io/projected/99d9b571-c89b-4520-b2f2-f0a790f5985b-kube-api-access-x5624\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:34.178698 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.178637    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99d9b571-c89b-4520-b2f2-f0a790f5985b-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:34.178698 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.178646    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d9b571-c89b-4520-b2f2-f0a790f5985b-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:34.732683 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.732641    2578 generic.go:356] "Generic (PLEG): container finished" podID="99d9b571-c89b-4520-b2f2-f0a790f5985b" containerID="264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8" exitCode=0
Jul 27 22:02:34.733102 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.732704    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"
Jul 27 22:02:34.733102 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.732708    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z" event={"ID":"99d9b571-c89b-4520-b2f2-f0a790f5985b","Type":"ContainerDied","Data":"264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8"}
Jul 27 22:02:34.733102 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.732790    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z" event={"ID":"99d9b571-c89b-4520-b2f2-f0a790f5985b","Type":"ContainerDied","Data":"e5f5237ad5fdb8b109953654f15f9888b33d4326dd29c45e245634b630995584"}
Jul 27 22:02:34.733102 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.732821    2578 scope.go:117] "RemoveContainer" containerID="264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8"
Jul 27 22:02:34.741438 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.741420    2578 scope.go:117] "RemoveContainer" containerID="264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8"
Jul 27 22:02:34.741717 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:34.741699    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8\": container with ID starting with 264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8 not found: ID does not exist" containerID="264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8"
Jul 27 22:02:34.741804 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.741749    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8"} err="failed to get container status \"264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8\": rpc error: code = NotFound desc = could not find container \"264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8\": container with ID starting with 264058d45aceafedcdcfd18ef1eaeae000f2bb850680e0bf82dd0ba46ecad8a8 not found: ID does not exist"
Jul 27 22:02:34.768168 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.768139    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"]
Jul 27 22:02:34.777966 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:34.777936    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58e80fe2-856f74956f-5nh5z"]
Jul 27 22:02:35.204747 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:35.204701    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"]
Jul 27 22:02:35.205011 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:35.204966    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy" containerID="cri-o://35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251" gracePeriod=30
Jul 27 22:02:35.605598 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:35.605511    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d9b571-c89b-4520-b2f2-f0a790f5985b" path="/var/lib/kubelet/pods/99d9b571-c89b-4520-b2f2-f0a790f5985b/volumes"
Jul 27 22:02:37.961837 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:37.961799    2578 patch_prober.go:28] interesting pod/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 27 22:02:37.962233 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:37.961862    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 22:02:38.448186 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448154    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"]
Jul 27 22:02:38.448491 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448479    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy"
Jul 27 22:02:38.448544 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448492    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy"
Jul 27 22:02:38.448544 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448502    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy"
Jul 27 22:02:38.448544 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448508    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy"
Jul 27 22:02:38.448544 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448520    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d9b571-c89b-4520-b2f2-f0a790f5985b" containerName="maas-api"
Jul 27 22:02:38.448544 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448528    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d9b571-c89b-4520-b2f2-f0a790f5985b" containerName="maas-api"
Jul 27 22:02:38.448701 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448588    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d9b571-c89b-4520-b2f2-f0a790f5985b" containerName="maas-api"
Jul 27 22:02:38.448701 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448596    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy"
Jul 27 22:02:38.448701 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.448601    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5adced16-de7c-4de3-b8b3-5a767df1a725" containerName="istio-proxy"
Jul 27 22:02:38.453051 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.453030    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:38.457964 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.457946    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-52fa39dc-serving-cert\""
Jul 27 22:02:38.482690 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.482655    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"]
Jul 27 22:02:38.623162 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.623129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfvgd\" (UniqueName: \"kubernetes.io/projected/d71401ab-6907-43e6-afeb-9ae241c0bb92-kube-api-access-pfvgd\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:38.623326 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.623172    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71401ab-6907-43e6-afeb-9ae241c0bb92-openshift-service-ca\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:38.623326 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.623212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71401ab-6907-43e6-afeb-9ae241c0bb92-maas-api-tls\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:38.724534 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.724452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71401ab-6907-43e6-afeb-9ae241c0bb92-maas-api-tls\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:38.724534 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.724514    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfvgd\" (UniqueName: \"kubernetes.io/projected/d71401ab-6907-43e6-afeb-9ae241c0bb92-kube-api-access-pfvgd\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:38.724746 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.724545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71401ab-6907-43e6-afeb-9ae241c0bb92-openshift-service-ca\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:38.724746 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:38.724611    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-serving-cert: secret "maas-api-e2e-derive-52fa39dc-serving-cert" not found
Jul 27 22:02:38.724746 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:38.724678    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71401ab-6907-43e6-afeb-9ae241c0bb92-maas-api-tls podName:d71401ab-6907-43e6-afeb-9ae241c0bb92 nodeName:}" failed. No retries permitted until 2026-07-27 22:02:39.224660839 +0000 UTC m=+1730.120470267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d71401ab-6907-43e6-afeb-9ae241c0bb92-maas-api-tls") pod "maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll" (UID: "d71401ab-6907-43e6-afeb-9ae241c0bb92") : secret "maas-api-e2e-derive-52fa39dc-serving-cert" not found
Jul 27 22:02:38.730715 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.725606    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71401ab-6907-43e6-afeb-9ae241c0bb92-openshift-service-ca\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:38.749450 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:38.749420    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfvgd\" (UniqueName: \"kubernetes.io/projected/d71401ab-6907-43e6-afeb-9ae241c0bb92-kube-api-access-pfvgd\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:39.228639 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:39.228601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71401ab-6907-43e6-afeb-9ae241c0bb92-maas-api-tls\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:39.231033 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:39.231014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71401ab-6907-43e6-afeb-9ae241c0bb92-maas-api-tls\") pod \"maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:39.363097 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:39.363056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:39.532639 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:39.532615    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"]
Jul 27 22:02:39.534170 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:02:39.534143    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd71401ab_6907_43e6_afeb_9ae241c0bb92.slice/crio-01bdbf4712c5f56234b9b3abf598181d888edb0d3487d19fa6ce017790ac880b WatchSource:0}: Error finding container 01bdbf4712c5f56234b9b3abf598181d888edb0d3487d19fa6ce017790ac880b: Status 404 returned error can't find the container with id 01bdbf4712c5f56234b9b3abf598181d888edb0d3487d19fa6ce017790ac880b
Jul 27 22:02:39.759023 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:39.758987    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll" event={"ID":"d71401ab-6907-43e6-afeb-9ae241c0bb92","Type":"ContainerStarted","Data":"01bdbf4712c5f56234b9b3abf598181d888edb0d3487d19fa6ce017790ac880b"}
Jul 27 22:02:40.457588 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.457565    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:40.642650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642573    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-data\") pod \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") "
Jul 27 22:02:40.642650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642609    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-credential-socket\") pod \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") "
Jul 27 22:02:40.642650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642630    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-envoy\") pod \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") "
Jul 27 22:02:40.642952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642676    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istiod-ca-cert\") pod \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") "
Jul 27 22:02:40.642952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642762    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-token\") pod \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") "
Jul 27 22:02:40.642952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642786    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-socket\") pod \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") "
Jul 27 22:02:40.642952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642821    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-certs\") pod \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") "
Jul 27 22:02:40.642952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642874    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-podinfo\") pod \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") "
Jul 27 22:02:40.642952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642900    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n72bf\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-kube-api-access-n72bf\") pod \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\" (UID: \"652132ce-5416-4c89-9ca7-ef8a74d15fd2\") "
Jul 27 22:02:40.643257 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.642923    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "652132ce-5416-4c89-9ca7-ef8a74d15fd2" (UID: "652132ce-5416-4c89-9ca7-ef8a74d15fd2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:40.643257 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.643077    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "652132ce-5416-4c89-9ca7-ef8a74d15fd2" (UID: "652132ce-5416-4c89-9ca7-ef8a74d15fd2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:02:40.643257 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.643186    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "652132ce-5416-4c89-9ca7-ef8a74d15fd2" (UID: "652132ce-5416-4c89-9ca7-ef8a74d15fd2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:40.643257 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.643210    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:40.643257 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.643228    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:40.643257 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.643237    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "652132ce-5416-4c89-9ca7-ef8a74d15fd2" (UID: "652132ce-5416-4c89-9ca7-ef8a74d15fd2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:40.643749 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.643438    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-data" (OuterVolumeSpecName: "istio-data") pod "652132ce-5416-4c89-9ca7-ef8a74d15fd2" (UID: "652132ce-5416-4c89-9ca7-ef8a74d15fd2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:40.645243 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.645214    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "652132ce-5416-4c89-9ca7-ef8a74d15fd2" (UID: "652132ce-5416-4c89-9ca7-ef8a74d15fd2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:02:40.645243 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.645236    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-kube-api-access-n72bf" (OuterVolumeSpecName: "kube-api-access-n72bf") pod "652132ce-5416-4c89-9ca7-ef8a74d15fd2" (UID: "652132ce-5416-4c89-9ca7-ef8a74d15fd2"). InnerVolumeSpecName "kube-api-access-n72bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:40.645518 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.645497    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "652132ce-5416-4c89-9ca7-ef8a74d15fd2" (UID: "652132ce-5416-4c89-9ca7-ef8a74d15fd2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:40.645518 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.645509    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-token" (OuterVolumeSpecName: "istio-token") pod "652132ce-5416-4c89-9ca7-ef8a74d15fd2" (UID: "652132ce-5416-4c89-9ca7-ef8a74d15fd2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:40.744263 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.744244    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:40.744263 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.744264    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:40.744385 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.744272    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:40.744385 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.744280    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:40.744385 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.744289    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/652132ce-5416-4c89-9ca7-ef8a74d15fd2-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:40.744385 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.744297    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/652132ce-5416-4c89-9ca7-ef8a74d15fd2-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:40.744385 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.744306    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n72bf\" (UniqueName: \"kubernetes.io/projected/652132ce-5416-4c89-9ca7-ef8a74d15fd2-kube-api-access-n72bf\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:40.765404 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.765376    2578 generic.go:356] "Generic (PLEG): container finished" podID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerID="35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251" exitCode=0
Jul 27 22:02:40.765495 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.765428    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"
Jul 27 22:02:40.765495 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.765457    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" event={"ID":"652132ce-5416-4c89-9ca7-ef8a74d15fd2","Type":"ContainerDied","Data":"35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251"}
Jul 27 22:02:40.765586 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.765508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8" event={"ID":"652132ce-5416-4c89-9ca7-ef8a74d15fd2","Type":"ContainerDied","Data":"e943b161942ceddd0527806db06ebffb966ca06c6eaefd60ea58f080feecf661"}
Jul 27 22:02:40.765586 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.765535    2578 scope.go:117] "RemoveContainer" containerID="35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251"
Jul 27 22:02:40.767104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.767081    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll" event={"ID":"d71401ab-6907-43e6-afeb-9ae241c0bb92","Type":"ContainerStarted","Data":"482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f"}
Jul 27 22:02:40.767218 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.767202    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:40.774778 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.774762    2578 scope.go:117] "RemoveContainer" containerID="bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30"
Jul 27 22:02:40.782545 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.782530    2578 scope.go:117] "RemoveContainer" containerID="35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251"
Jul 27 22:02:40.782805 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:40.782786    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251\": container with ID starting with 35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251 not found: ID does not exist" containerID="35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251"
Jul 27 22:02:40.782865 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.782813    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251"} err="failed to get container status \"35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251\": rpc error: code = NotFound desc = could not find container \"35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251\": container with ID starting with 35c0d4dd054f9010743921fbdec99f4240c35498a970f02067759449ac586251 not found: ID does not exist"
Jul 27 22:02:40.782865 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.782831    2578 scope.go:117] "RemoveContainer" containerID="bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30"
Jul 27 22:02:40.783040 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:40.783025    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30\": container with ID starting with bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30 not found: ID does not exist" containerID="bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30"
Jul 27 22:02:40.783077 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.783045    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30"} err="failed to get container status \"bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30\": rpc error: code = NotFound desc = could not find container \"bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30\": container with ID starting with bcf890fef03457423ab7bb84e3a13e72796400c0675c24aa5cbf1eb740881b30 not found: ID does not exist"
Jul 27 22:02:40.833688 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.833667    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"]
Jul 27 22:02:40.849885 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.849864    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-58e80fe2-openshift-default-8559b768f4-cvjb8"]
Jul 27 22:02:40.918135 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:40.918095    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll" podStartSLOduration=2.5026497279999997 podStartE2EDuration="2.918081526s" totalImagesPullingTime="415.431798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:40.916381946 +0000 UTC m=+1731.812191396" watchObservedRunningTime="2026-07-27 22:02:40.918081526 +0000 UTC m=+1731.813890977"
Jul 27 22:02:41.606004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:41.605971    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" path="/var/lib/kubelet/pods/652132ce-5416-4c89-9ca7-ef8a74d15fd2/volumes"
Jul 27 22:02:46.779093 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:46.779062    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:52.988402 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:52.988371    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"]
Jul 27 22:02:52.988775 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:52.988720    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy"
Jul 27 22:02:52.988775 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:52.988749    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy"
Jul 27 22:02:52.988775 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:52.988764    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy"
Jul 27 22:02:52.988775 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:52.988769    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy"
Jul 27 22:02:52.988916 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:52.988835    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy"
Jul 27 22:02:52.988916 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:52.988844    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="652132ce-5416-4c89-9ca7-ef8a74d15fd2" containerName="istio-proxy"
Jul 27 22:02:52.992839 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:52.992816    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"
Jul 27 22:02:52.996358 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:52.996340    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxwpp\""
Jul 27 22:02:53.023822 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:53.023797    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"]
Jul 27 22:02:53.134248 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:53.134221    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fed919-288b-466e-b736-6143d3b02e6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx\" (UID: \"a9fed919-288b-466e-b736-6143d3b02e6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"
Jul 27 22:02:53.235482 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:53.235456    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fed919-288b-466e-b736-6143d3b02e6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx\" (UID: \"a9fed919-288b-466e-b736-6143d3b02e6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"
Jul 27 22:02:53.236043 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:53.236027    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fed919-288b-466e-b736-6143d3b02e6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx\" (UID: \"a9fed919-288b-466e-b736-6143d3b02e6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"
Jul 27 22:02:53.303605 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:53.303551    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"
Jul 27 22:02:53.456411 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:53.456389    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"]
Jul 27 22:02:53.458333 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:02:53.458300    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fed919_288b_466e_b736_6143d3b02e6f.slice/crio-e0e290a2eeab0ada1ab7529e646eb1feb9f14a96016b3f183b91602c63f4d0cf WatchSource:0}: Error finding container e0e290a2eeab0ada1ab7529e646eb1feb9f14a96016b3f183b91602c63f4d0cf: Status 404 returned error can't find the container with id e0e290a2eeab0ada1ab7529e646eb1feb9f14a96016b3f183b91602c63f4d0cf
Jul 27 22:02:53.829059 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:53.829024    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx" event={"ID":"a9fed919-288b-466e-b736-6143d3b02e6f","Type":"ContainerStarted","Data":"f2c835ddcdea50adcb8017e9ccd3b0b2eda09e3c6f9c6b365ac55fbe9e07446d"}
Jul 27 22:02:53.829162 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:53.829082    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx" event={"ID":"a9fed919-288b-466e-b736-6143d3b02e6f","Type":"ContainerStarted","Data":"e0e290a2eeab0ada1ab7529e646eb1feb9f14a96016b3f183b91602c63f4d0cf"}
Jul 27 22:02:53.857609 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:53.857566    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx" podStartSLOduration=1.857551565 podStartE2EDuration="1.857551565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:02:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:53.856138019 +0000 UTC m=+1744.751947473" watchObservedRunningTime="2026-07-27 22:02:53.857551565 +0000 UTC m=+1744.753361013"
Jul 27 22:02:54.835344 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:54.835300    2578 generic.go:356] "Generic (PLEG): container finished" podID="a9fed919-288b-466e-b736-6143d3b02e6f" containerID="f2c835ddcdea50adcb8017e9ccd3b0b2eda09e3c6f9c6b365ac55fbe9e07446d" exitCode=0
Jul 27 22:02:54.835689 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:54.835380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx" event={"ID":"a9fed919-288b-466e-b736-6143d3b02e6f","Type":"ContainerDied","Data":"f2c835ddcdea50adcb8017e9ccd3b0b2eda09e3c6f9c6b365ac55fbe9e07446d"}
Jul 27 22:02:55.967455 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:55.967426    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"
Jul 27 22:02:56.061491 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:56.061466    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fed919-288b-466e-b736-6143d3b02e6f-maas-api-service-ca\") pod \"a9fed919-288b-466e-b736-6143d3b02e6f\" (UID: \"a9fed919-288b-466e-b736-6143d3b02e6f\") "
Jul 27 22:02:56.061832 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:56.061802    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9fed919-288b-466e-b736-6143d3b02e6f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a9fed919-288b-466e-b736-6143d3b02e6f" (UID: "a9fed919-288b-466e-b736-6143d3b02e6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:02:56.162686 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:56.162662    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fed919-288b-466e-b736-6143d3b02e6f-maas-api-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:56.846756 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:56.846706    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"
Jul 27 22:02:56.846930 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:56.846723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx" event={"ID":"a9fed919-288b-466e-b736-6143d3b02e6f","Type":"ContainerDied","Data":"e0e290a2eeab0ada1ab7529e646eb1feb9f14a96016b3f183b91602c63f4d0cf"}
Jul 27 22:02:56.846930 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:56.846802    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0e290a2eeab0ada1ab7529e646eb1feb9f14a96016b3f183b91602c63f4d0cf"
Jul 27 22:02:59.527939 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.527904    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"]
Jul 27 22:02:59.528336 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.528090    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll" podUID="d71401ab-6907-43e6-afeb-9ae241c0bb92" containerName="maas-api" containerID="cri-o://482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f" gracePeriod=30
Jul 27 22:02:59.784701 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.784645    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:59.788720 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.788704    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71401ab-6907-43e6-afeb-9ae241c0bb92-maas-api-tls\") pod \"d71401ab-6907-43e6-afeb-9ae241c0bb92\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") "
Jul 27 22:02:59.788791 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.788777    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71401ab-6907-43e6-afeb-9ae241c0bb92-openshift-service-ca\") pod \"d71401ab-6907-43e6-afeb-9ae241c0bb92\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") "
Jul 27 22:02:59.788924 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.788907    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfvgd\" (UniqueName: \"kubernetes.io/projected/d71401ab-6907-43e6-afeb-9ae241c0bb92-kube-api-access-pfvgd\") pod \"d71401ab-6907-43e6-afeb-9ae241c0bb92\" (UID: \"d71401ab-6907-43e6-afeb-9ae241c0bb92\") "
Jul 27 22:02:59.789128 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.789107    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d71401ab-6907-43e6-afeb-9ae241c0bb92-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d71401ab-6907-43e6-afeb-9ae241c0bb92" (UID: "d71401ab-6907-43e6-afeb-9ae241c0bb92"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:02:59.789208 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.789187    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71401ab-6907-43e6-afeb-9ae241c0bb92-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:59.791192 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.791167    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d71401ab-6907-43e6-afeb-9ae241c0bb92-kube-api-access-pfvgd" (OuterVolumeSpecName: "kube-api-access-pfvgd") pod "d71401ab-6907-43e6-afeb-9ae241c0bb92" (UID: "d71401ab-6907-43e6-afeb-9ae241c0bb92"). InnerVolumeSpecName "kube-api-access-pfvgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:59.791301 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.791218    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d71401ab-6907-43e6-afeb-9ae241c0bb92-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d71401ab-6907-43e6-afeb-9ae241c0bb92" (UID: "d71401ab-6907-43e6-afeb-9ae241c0bb92"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:02:59.861989 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.861963    2578 generic.go:356] "Generic (PLEG): container finished" podID="d71401ab-6907-43e6-afeb-9ae241c0bb92" containerID="482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f" exitCode=0
Jul 27 22:02:59.862137 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.862006    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"
Jul 27 22:02:59.862137 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.862051    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll" event={"ID":"d71401ab-6907-43e6-afeb-9ae241c0bb92","Type":"ContainerDied","Data":"482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f"}
Jul 27 22:02:59.862137 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.862109    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll" event={"ID":"d71401ab-6907-43e6-afeb-9ae241c0bb92","Type":"ContainerDied","Data":"01bdbf4712c5f56234b9b3abf598181d888edb0d3487d19fa6ce017790ac880b"}
Jul 27 22:02:59.862310 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.862139    2578 scope.go:117] "RemoveContainer" containerID="482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f"
Jul 27 22:02:59.871635 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.871612    2578 scope.go:117] "RemoveContainer" containerID="482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f"
Jul 27 22:02:59.871957 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:02:59.871931    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f\": container with ID starting with 482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f not found: ID does not exist" containerID="482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f"
Jul 27 22:02:59.872048 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.871967    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f"} err="failed to get container status \"482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f\": rpc error: code = NotFound desc = could not find container \"482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f\": container with ID starting with 482ea15e13b8c10f7b6f756a2ba94fd04e4d55b70c96adb5c0cde0ab9b84848f not found: ID does not exist"
Jul 27 22:02:59.889781 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.889758    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71401ab-6907-43e6-afeb-9ae241c0bb92-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:59.889781 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.889779    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfvgd\" (UniqueName: \"kubernetes.io/projected/d71401ab-6907-43e6-afeb-9ae241c0bb92-kube-api-access-pfvgd\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:02:59.900587 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.900560    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"]
Jul 27 22:02:59.913992 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:02:59.913968    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-52fa39dc-5cf74cb6f8-hwsll"]
Jul 27 22:03:00.033053 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:03:00.033027    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"]
Jul 27 22:03:00.040773 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:03:00.040697    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-52fa39dc-5dlpx"]
Jul 27 22:03:01.605619 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:03:01.605580    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9fed919-288b-466e-b736-6143d3b02e6f" path="/var/lib/kubelet/pods/a9fed919-288b-466e-b736-6143d3b02e6f/volumes"
Jul 27 22:03:01.606090 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:03:01.605970    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d71401ab-6907-43e6-afeb-9ae241c0bb92" path="/var/lib/kubelet/pods/d71401ab-6907-43e6-afeb-9ae241c0bb92/volumes"
Jul 27 22:03:49.594325 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:03:49.594292    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:03:49.600049 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:03:49.600020    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:04:45.586232 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.586197    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"]
Jul 27 22:04:45.586720 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.586549    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9fed919-288b-466e-b736-6143d3b02e6f" containerName="revoke-keys"
Jul 27 22:04:45.586720 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.586561    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9fed919-288b-466e-b736-6143d3b02e6f" containerName="revoke-keys"
Jul 27 22:04:45.586720 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.586585    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d71401ab-6907-43e6-afeb-9ae241c0bb92" containerName="maas-api"
Jul 27 22:04:45.586720 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.586590    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d71401ab-6907-43e6-afeb-9ae241c0bb92" containerName="maas-api"
Jul 27 22:04:45.586720 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.586638    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d71401ab-6907-43e6-afeb-9ae241c0bb92" containerName="maas-api"
Jul 27 22:04:45.586720 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.586647    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9fed919-288b-466e-b736-6143d3b02e6f" containerName="revoke-keys"
Jul 27 22:04:45.589493 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.589477    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.592950 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.592926    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d69a5dde-openshift-default-dockercfg-26gpx\""
Jul 27 22:04:45.726401 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.726372    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-envoy\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.726556 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.726420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-credential-socket\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.726556 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.726441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-data\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.726556 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.726456    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-podinfo\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.726556 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.726480    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-certs\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.726556 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.726501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-token\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.726556 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.726520    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldmlj\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-kube-api-access-ldmlj\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.726556 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.726553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-socket\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.726841 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.726574    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istiod-ca-cert\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827036 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-credential-socket\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827036 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-data\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827229 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-podinfo\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827229 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-certs\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827229 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-token\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827229 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827227    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmlj\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-kube-api-access-ldmlj\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827425 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-socket\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827425 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istiod-ca-cert\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827425 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827367    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-envoy\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827581 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-credential-socket\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827581 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827512    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-certs\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827691 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827587    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-socket\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.827691 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-data\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.828018 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.827994    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istiod-ca-cert\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.829671 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.829652    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-envoy\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:45.829934 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:04:45.829915    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-podinfo\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:04:58.494277 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:04:58.494225    2578 controller.go:195] "Failed to update lease" err="Put \"https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-161.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 27 22:05:08.494445 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:08.494403    2578 controller.go:195] "Failed to update lease" err="Put \"https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-161.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 27 22:05:18.495169 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:18.495119    2578 controller.go:195] "Failed to update lease" err="Put \"https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-161.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 27 22:05:27.669062 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:27.669017    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T22:05:17Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T22:05:17Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T22:05:17Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T22:05:17Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-132-161.ec2.internal\": Patch \"https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-132-161.ec2.internal/status?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 27 22:05:28.495368 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:28.495328    2578 controller.go:195] "Failed to update lease" err="Put \"https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-161.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 27 22:05:30.599104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599068    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-42c4q\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599066    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599082    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599108    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wbhc9\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599092    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599130    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599132    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599149    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599156    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599168    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6w5sb\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599183    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m5fqm\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599214    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599216    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599232    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599241    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599252    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599255    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599274    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599283    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599290    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599301    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"default-dockercfg-dwq99\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.599826 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599313    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.600694 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:30.599305    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"1d1fad47-f4dd-4033-a2f0-839bff4a162a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-27T22:04:45Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-27T22:04:45Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-27T22:04:45Z\\\",\\\"message\\\":\\\"containers with unready status: [istio-proxy]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-27T22:04:45Z\\\",\\\"message\\\":\\\"containers with unready status: [istio-proxy]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-ldmlj\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.132.161\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.132.161\\\"}],\\\"observedGeneration\\\":1,\\\"startTime\\\":\\\"2026-07-27T22:04:45Z\\\"}}\" for pod \"openshift-ingress\"/\"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\": Patch \"https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2/status\": http2: client connection lost" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:05:30.600694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599362    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f5bdw\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.600694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599379    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.600694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599383    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8c4jf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.600694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599394    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.600694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599411    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pfmkx\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.600694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599432    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9k8kt\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.600694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599434    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.600694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599445    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:30.599459    2578 projected.go:196] Error preparing data for projected volume kube-api-access-ldmlj for pod openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2: failed to fetch token: Post "https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/serviceaccounts/e2e-mt-d69a5dde-openshift-default/token": http2: client connection lost
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599463    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599471    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2mvn4\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599486    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599493    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599464    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599521    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599521    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:30.599536    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-kube-api-access-ldmlj podName:1d1fad47-f4dd-4033-a2f0-839bff4a162a nodeName:}" failed. No retries permitted until 2026-07-27 22:05:31.099515412 +0000 UTC m=+1901.995324845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ldmlj" (UniqueName: "kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-kube-api-access-ldmlj") pod "e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a") : failed to fetch token: Post "https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/serviceaccounts/e2e-mt-d69a5dde-openshift-default/token": http2: client connection lost
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599531    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599544    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:30.599555    2578 projected.go:196] Error preparing data for projected volume istio-token for pod openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2: failed to fetch token: Post "https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/serviceaccounts/e2e-mt-d69a5dde-openshift-default/token": http2: client connection lost
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599562    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599565    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599595    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.601253 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599601    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599526    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"default-dockercfg-l6bvf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:30.599617    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-token podName:1d1fad47-f4dd-4033-a2f0-839bff4a162a nodeName:}" failed. No retries permitted until 2026-07-27 22:05:31.099605708 +0000 UTC m=+1901.995415142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-token" (UniqueName: "kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-token") pod "e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a") : failed to fetch token: Post "https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/serviceaccounts/e2e-mt-d69a5dde-openshift-default/token": http2: client connection lost
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599601    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-txdpd\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599581    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599637    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599642    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599657    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z2vzc\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599662    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599673    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599676    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599683    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599683    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599693    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599698    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599662    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g5hgq\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599709    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599723    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pj4wj\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599771    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599776    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599784    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599791    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599791    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-dbhm6\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599816    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599829    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xl2cn\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599835    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599849    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599856    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599862    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599867    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599866    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599890    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599900    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599906    2578 reflector.go:571] "Warning: watch ended with error" reflector="pkg/kubelet/config/apiserver.go:66" type="*v1.Pod" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599905    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.602984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599924    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599935    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599963    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599971    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599350    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.599997    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600003    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcmmw\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600007    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600017    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600032    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600033    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-26t5b\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600059    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600068    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600071    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x6q7n\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600098    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600112    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bzc5h\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600111    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xnkh\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600114    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.603649 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600138    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600142    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600148    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-brxjg\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600160    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-s9j86\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600167    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600170    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"original-pull-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600185    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kb678\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600186    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600196    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7fmv4\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600211    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-667rr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600216    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600232    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600238    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zzpc9\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600252    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600257    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600268    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600274    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600276    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.604222 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600293    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600306    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600317    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600321    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"default-dockercfg-zv6n8\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600344    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600345    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ghxlq\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600345    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600386    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:30.600395    2578 controller.go:195] "Failed to update lease" err="Put \"https://a1975e2ca7f6c41d999b9a439c0ec849-0e8d6d289eca585a.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-161.ec2.internal?timeout=10s\": http2: client connection lost"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600410    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600422    2578 controller.go:115] "failed to update lease using latest lease, fallback to ensure lease" err="failed 5 attempts to update lease"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600433    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600449    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600467    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600471    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600494    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-p9txj\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600499    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600422    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600508    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"cacerts\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605022 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600527    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600534    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600531    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600536    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600549    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600556    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600559    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600570    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600576    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600593    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600594    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9c4jr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600595    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600603    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d6xkx\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600398    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fsgcq\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600629    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-dockercfg-bpkp7\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:30.605650 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:30.600611    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 22:05:31.162134 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.162105    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-token\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:05:31.162298 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.162141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmlj\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-kube-api-access-ldmlj\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:05:31.181946 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.181916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-token\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:05:31.182072 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.181985    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldmlj\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-kube-api-access-ldmlj\") pod \"e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") " pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:05:31.202986 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.202963    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d69a5dde-openshift-default-dockercfg-26gpx\""
Jul 27 22:05:31.210692 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.210675    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:05:31.386232 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:05:31.386194    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d1fad47_f4dd_4033_a2f0_839bff4a162a.slice/crio-ceaa4012b2b6f148c4b1fc423ba530a18c008b9a8bf50e694e1389790692667d WatchSource:0}: Error finding container ceaa4012b2b6f148c4b1fc423ba530a18c008b9a8bf50e694e1389790692667d: Status 404 returned error can't find the container with id ceaa4012b2b6f148c4b1fc423ba530a18c008b9a8bf50e694e1389790692667d
Jul 27 22:05:31.388265 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.388230    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:05:31.388359 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.388299    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:05:31.388359 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.388327    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:05:31.415231 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.415180    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g5hgq\""
Jul 27 22:05:31.418352 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.418338    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.425271 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.425252    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.425693 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.425677    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 22:05:31.427061 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.427042    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f5bdw\""
Jul 27 22:05:31.435223 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.435207    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.443862 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.443843    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 27 22:05:31.450899 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.450879    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.472675 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.472656    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.476984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.476958    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.481620 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.481603    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.488630 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.488612    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 22:05:31.488803 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.488789    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.497437 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.497417    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 27 22:05:31.497437 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.497431    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.498452 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.498429    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 22:05:31.510875 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.510856    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 22:05:31.515252 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.515238    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pj4wj\""
Jul 27 22:05:31.515335 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.515323    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pfmkx\""
Jul 27 22:05:31.516194 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.516177    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 27 22:05:31.519856 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.519837    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 22:05:31.526320 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.526294    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-42c4q\""
Jul 27 22:05:31.544062 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.544042    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.560440 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.560422    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 27 22:05:31.565049 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.565035    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 22:05:31.566102 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.566085    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 22:05:31.571322 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.571307    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d6xkx\""
Jul 27 22:05:31.580360 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.580340    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9c4jr\""
Jul 27 22:05:31.585210 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.585195    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.595962 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.595943    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 22:05:31.596070 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.596055    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7fmv4\""
Jul 27 22:05:31.596372 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.596360    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 22:05:31.596704 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.596689    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.597351 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.597329    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.597451 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.597332    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.598239 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.598214    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" event={"ID":"1d1fad47-f4dd-4033-a2f0-839bff4a162a","Type":"ContainerStarted","Data":"af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655"}
Jul 27 22:05:31.598336 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.598276    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" event={"ID":"1d1fad47-f4dd-4033-a2f0-839bff4a162a","Type":"ContainerStarted","Data":"ceaa4012b2b6f148c4b1fc423ba530a18c008b9a8bf50e694e1389790692667d"}
Jul 27 22:05:31.603152 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.603136    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 22:05:31.603460 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.603169    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 22:05:31.603460 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.603183    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeHasSufficientPID"
Jul 27 22:05:31.603460 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.603199    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-161.ec2.internal" event="NodeReady"
Jul 27 22:05:31.609004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.608988    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.611395 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.611378    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bzc5h\""
Jul 27 22:05:31.612420 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.612405    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 22:05:31.622690 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.622673    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 27 22:05:31.628478 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.628455    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 22:05:31.633548 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.633518    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 22:05:31.635934 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.635917    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.639923 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.639904    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.642451 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.642435    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 22:05:31.647136 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.647117    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 27 22:05:31.657446 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.657426    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 22:05:31.658864 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.658847    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.665840 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.665777    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.668397 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.668380    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ghxlq\""
Jul 27 22:05:31.670189 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.670175    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6w5sb\""
Jul 27 22:05:31.676273 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.676255    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-p9txj\""
Jul 27 22:05:31.690027 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.690008    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 22:05:31.690302 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.690284    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zzpc9\""
Jul 27 22:05:31.690609 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.690596    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wbhc9\""
Jul 27 22:05:31.693978 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.693957    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.703605 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.703589    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 27 22:05:31.708211 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.708196    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 22:05:31.727469 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.727451    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.729646 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.729633    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 27 22:05:31.732327 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.732313    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.732899 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.732887    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z2vzc\""
Jul 27 22:05:31.737291 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.737266    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.738265 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.738249    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 22:05:31.764827 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.764806    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 22:05:31.764966 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.764949    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.765026 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.764953    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 22:05:31.770232 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.770206    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 22:05:31.770672 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.770647    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 22:05:31.773171 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.773120    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" podStartSLOduration=46.773095448 podStartE2EDuration="46.773095448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:04:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:05:31.652630644 +0000 UTC m=+1902.548440094" watchObservedRunningTime="2026-07-27 22:05:31.773095448 +0000 UTC m=+1902.668904901"
Jul 27 22:05:31.775169 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.775148    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"]
Jul 27 22:05:31.780241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.780221    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.782850 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.782833    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-26t5b\""
Jul 27 22:05:31.794402 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.794387    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kb678\""
Jul 27 22:05:31.796537 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.796514    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-s9j86\""
Jul 27 22:05:31.800209 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.800192    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8c4jf\""
Jul 27 22:05:31.806452 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.806434    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 22:05:31.808877 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.808862    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.817848 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.817830    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 27 22:05:31.822221 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.822197    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 27 22:05:31.824652 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.824635    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.829299 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.829284    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.829950 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.829934    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-667rr\""
Jul 27 22:05:31.831660 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.831647    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.831929 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.831916    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.832241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.832224    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 22:05:31.839939 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.839924    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.842278 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.842265    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.845640 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.845627    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.851029 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.851013    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fsgcq\""
Jul 27 22:05:31.854560 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.854547    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.856180 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.856166    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x6q7n\""
Jul 27 22:05:31.859306 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.859290    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.860869 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.860857    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 22:05:31.867612 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.867599    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xnkh\""
Jul 27 22:05:31.878098 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.878079    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.878238 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.878224    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 22:05:31.878540 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.878528    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 27 22:05:31.885343 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.885330    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 22:05:31.887992 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.887969    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcmmw\""
Jul 27 22:05:31.892578 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.892563    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 27 22:05:31.906039 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.906023    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dwq99\""
Jul 27 22:05:31.917570 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.917512    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 27 22:05:31.929228 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.929202    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9k8kt\""
Jul 27 22:05:31.952545 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.952518    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 22:05:31.959491 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.959474    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 22:05:31.972676 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.972652    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 27 22:05:31.972825 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.972809    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 27 22:05:31.978099 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.978083    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-brxjg\""
Jul 27 22:05:31.978262 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.978250    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 27 22:05:31.982781 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.982764    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m5fqm\""
Jul 27 22:05:31.984967 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.984952    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-txdpd\""
Jul 27 22:05:31.986636 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.986622    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 22:05:31.994856 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.994839    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 27 22:05:31.997356 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:31.997340    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 27 22:05:32.003746 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.003713    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 22:05:32.008923 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.008909    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 22:05:32.009041 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.009027    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2mvn4\""
Jul 27 22:05:32.015491 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.015476    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 22:05:32.030548 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.030529    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-l6bvf\""
Jul 27 22:05:32.039070 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.039055    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bpkp7\""
Jul 27 22:05:32.048562 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.048543    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 22:05:32.058021 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.058004    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 22:05:32.058123 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.058046    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 22:05:32.059173 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.059159    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 27 22:05:32.059637 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.059621    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 27 22:05:32.059720 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.059649    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 27 22:05:32.065592 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.065578    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 22:05:32.067776 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.067759    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 22:05:32.069081 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.069057    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 22:05:32.071413 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.071398    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xl2cn\""
Jul 27 22:05:32.092190 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.092169    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 22:05:32.092566 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.092553    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 22:05:32.104502 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.104486    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 22:05:32.112432 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.112414    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 22:05:32.117828 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.117812    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 22:05:32.134403 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.134385    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 27 22:05:32.154922 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.154905    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 27 22:05:32.177627 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.177577    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 22:05:32.194968 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.194948    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 22:05:32.211614 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.211588    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:05:32.213023 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.213003    2578 patch_prober.go:28] interesting pod/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 27 22:05:32.213092 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.213054    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" podUID="1d1fad47-f4dd-4033-a2f0-839bff4a162a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 27 22:05:32.213994 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.213981    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 22:05:32.238935 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.238915    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 22:05:32.255012 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.254991    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 27 22:05:32.280438 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.280417    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 22:05:32.294518 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.294497    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 22:05:32.314595 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.314573    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-dbhm6\""
Jul 27 22:05:32.333989 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.333965    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 22:05:32.356188 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:32.356170    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zv6n8\""
Jul 27 22:05:33.211992 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:33.211958    2578 patch_prober.go:28] interesting pod/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 27 22:05:33.212404 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:33.212018    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" podUID="1d1fad47-f4dd-4033-a2f0-839bff4a162a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 27 22:05:33.730691 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:33.730661    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"]
Jul 27 22:05:34.211546 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:34.211516    2578 patch_prober.go:28] interesting pod/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 27 22:05:34.211710 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:34.211578    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" podUID="1d1fad47-f4dd-4033-a2f0-839bff4a162a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 27 22:05:34.613981 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:34.613871    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" podUID="1d1fad47-f4dd-4033-a2f0-839bff4a162a" containerName="istio-proxy" containerID="cri-o://af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655" gracePeriod=30
Jul 27 22:05:39.886555 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.886387    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:05:39.923690 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.923656    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-envoy\") pod \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") "
Jul 27 22:05:39.923845 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.923708    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istiod-ca-cert\") pod \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") "
Jul 27 22:05:39.923845 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.923816    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldmlj\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-kube-api-access-ldmlj\") pod \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") "
Jul 27 22:05:39.923996 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.923848    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-socket\") pod \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") "
Jul 27 22:05:39.923996 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.923927    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-data\") pod \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") "
Jul 27 22:05:39.923996 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.923951    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-podinfo\") pod \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") "
Jul 27 22:05:39.923996 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.923978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-token\") pod \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") "
Jul 27 22:05:39.924204 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.924005    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-credential-socket\") pod \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") "
Jul 27 22:05:39.924204 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.924042    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-certs\") pod \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\" (UID: \"1d1fad47-f4dd-4033-a2f0-839bff4a162a\") "
Jul 27 22:05:39.924204 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.924151    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1d1fad47-f4dd-4033-a2f0-839bff4a162a" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:05:39.924382 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.924255    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-data" (OuterVolumeSpecName: "istio-data") pod "1d1fad47-f4dd-4033-a2f0-839bff4a162a" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:05:39.924382 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.924359    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:05:39.924488 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.924379    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:05:39.924488 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.924442    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1d1fad47-f4dd-4033-a2f0-839bff4a162a" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:05:39.924488 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.924461    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1d1fad47-f4dd-4033-a2f0-839bff4a162a" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:05:39.924753 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.924486    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1d1fad47-f4dd-4033-a2f0-839bff4a162a" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:05:39.926227 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.926197    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-kube-api-access-ldmlj" (OuterVolumeSpecName: "kube-api-access-ldmlj") pod "1d1fad47-f4dd-4033-a2f0-839bff4a162a" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a"). InnerVolumeSpecName "kube-api-access-ldmlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:05:39.926337 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.926272    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1d1fad47-f4dd-4033-a2f0-839bff4a162a" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:05:39.926337 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.926282    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1d1fad47-f4dd-4033-a2f0-839bff4a162a" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:05:39.926337 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:39.926292    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-token" (OuterVolumeSpecName: "istio-token") pod "1d1fad47-f4dd-4033-a2f0-839bff4a162a" (UID: "1d1fad47-f4dd-4033-a2f0-839bff4a162a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:05:40.024795 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.024764    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldmlj\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-kube-api-access-ldmlj\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:05:40.024795 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.024793    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:05:40.024987 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.024806    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:05:40.024987 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.024818    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:05:40.024987 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.024830    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:05:40.024987 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.024841    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:05:40.024987 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.024853    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d1fad47-f4dd-4033-a2f0-839bff4a162a-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:05:40.645252 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.645219    2578 generic.go:356] "Generic (PLEG): container finished" podID="1d1fad47-f4dd-4033-a2f0-839bff4a162a" containerID="af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655" exitCode=0
Jul 27 22:05:40.645465 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.645258    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"
Jul 27 22:05:40.645465 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.645296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" event={"ID":"1d1fad47-f4dd-4033-a2f0-839bff4a162a","Type":"ContainerDied","Data":"af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655"}
Jul 27 22:05:40.645465 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.645348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2" event={"ID":"1d1fad47-f4dd-4033-a2f0-839bff4a162a","Type":"ContainerDied","Data":"ceaa4012b2b6f148c4b1fc423ba530a18c008b9a8bf50e694e1389790692667d"}
Jul 27 22:05:40.645465 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.645377    2578 scope.go:117] "RemoveContainer" containerID="af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655"
Jul 27 22:05:40.656623 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.656602    2578 scope.go:117] "RemoveContainer" containerID="af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655"
Jul 27 22:05:40.656968 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:05:40.656944    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655\": container with ID starting with af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655 not found: ID does not exist" containerID="af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655"
Jul 27 22:05:40.657042 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:40.656981    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655"} err="failed to get container status \"af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655\": rpc error: code = NotFound desc = could not find container \"af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655\": container with ID starting with af74f824f3029f5d38c2386028b26a8f218daf63d5660540df4a794bf42ae655 not found: ID does not exist"
Jul 27 22:05:41.136297 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:41.136256    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"]
Jul 27 22:05:41.142848 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:41.142826    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d69a5dde-openshift-default-799f67b96-w86j2"]
Jul 27 22:05:41.606044 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:41.606006    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d1fad47-f4dd-4033-a2f0-839bff4a162a" path="/var/lib/kubelet/pods/1d1fad47-f4dd-4033-a2f0-839bff4a162a/volumes"
Jul 27 22:05:56.549474 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.549440    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"]
Jul 27 22:05:56.549890 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.549798    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d1fad47-f4dd-4033-a2f0-839bff4a162a" containerName="istio-proxy"
Jul 27 22:05:56.549890 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.549810    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d1fad47-f4dd-4033-a2f0-839bff4a162a" containerName="istio-proxy"
Jul 27 22:05:56.549890 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.549858    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d1fad47-f4dd-4033-a2f0-839bff4a162a" containerName="istio-proxy"
Jul 27 22:05:56.552593 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.552573    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.556261 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.556210    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e2733d0b-openshift-default-dockercfg-jmkps\""
Jul 27 22:05:56.575416 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.575391    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"]
Jul 27 22:05:56.650382 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.650342    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-data\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.650509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.650384    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-token\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.650509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.650431    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/348e3b15-86b9-4162-97b3-b8ef8f035f51-istiod-ca-cert\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.650509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.650477    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-socket\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.650509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.650505    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-266nw\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-kube-api-access-266nw\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.650675 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.650538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-certs\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.650675 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.650614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-credential-socket\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.650675 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.650647    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-podinfo\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.650812 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.650678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-envoy\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752019 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.751993    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-socket\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752161 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752026    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-266nw\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-kube-api-access-266nw\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752161 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-certs\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752161 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-credential-socket\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752161 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-podinfo\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752161 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-envoy\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752416 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-data\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752677 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752650    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-socket\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752770 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752718    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-data\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752904 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-token\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.752981 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.752887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-certs\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.753194 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.753173    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/348e3b15-86b9-4162-97b3-b8ef8f035f51-istiod-ca-cert\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.753477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.753169    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-credential-socket\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.755660 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.753554    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/348e3b15-86b9-4162-97b3-b8ef8f035f51-istiod-ca-cert\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.755811 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.755785    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-envoy\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.757629 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.757604    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-podinfo\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.763320 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.763297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-266nw\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-kube-api-access-266nw\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.765229 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.765209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-token\") pod \"e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") " pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:56.863436 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:56.863343    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:57.015668 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:57.015636    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"]
Jul 27 22:05:57.023168 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:05:57.023142    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod348e3b15_86b9_4162_97b3_b8ef8f035f51.slice/crio-ab0bebc84733582e0af859bec8acfe9784eb2e2050055edb9c3e5f3c7c57eb16 WatchSource:0}: Error finding container ab0bebc84733582e0af859bec8acfe9784eb2e2050055edb9c3e5f3c7c57eb16: Status 404 returned error can't find the container with id ab0bebc84733582e0af859bec8acfe9784eb2e2050055edb9c3e5f3c7c57eb16
Jul 27 22:05:57.025386 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:57.025356    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:05:57.025451 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:57.025425    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:05:57.025489 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:57.025454    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:05:57.728638 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:57.728601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" event={"ID":"348e3b15-86b9-4162-97b3-b8ef8f035f51","Type":"ContainerStarted","Data":"487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18"}
Jul 27 22:05:57.729114 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:57.728663    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" event={"ID":"348e3b15-86b9-4162-97b3-b8ef8f035f51","Type":"ContainerStarted","Data":"ab0bebc84733582e0af859bec8acfe9784eb2e2050055edb9c3e5f3c7c57eb16"}
Jul 27 22:05:57.760143 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:57.760089    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" podStartSLOduration=1.76007476 podStartE2EDuration="1.76007476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:05:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:05:57.759466375 +0000 UTC m=+1928.655275828" watchObservedRunningTime="2026-07-27 22:05:57.76007476 +0000 UTC m=+1928.655884210"
Jul 27 22:05:57.864377 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:57.864347    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:58.868315 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:58.868286    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:59.738004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:59.737978    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:59.738794 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:59.738775    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:05:59.955073 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:59.955042    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"]
Jul 27 22:05:59.958569 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:59.958548    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:05:59.961297 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:59.961276    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e2733d0b-serving-cert\""
Jul 27 22:05:59.970246 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:05:59.970223    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"]
Jul 27 22:06:00.079654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.079563    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55fa83b8-fbae-4d41-a268-9edad6589f09-maas-api-tls\") pod \"maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.079654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.079605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rwhq\" (UniqueName: \"kubernetes.io/projected/55fa83b8-fbae-4d41-a268-9edad6589f09-kube-api-access-9rwhq\") pod \"maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.079913 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.079716    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa83b8-fbae-4d41-a268-9edad6589f09-openshift-service-ca\") pod \"maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.180796 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.180769    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55fa83b8-fbae-4d41-a268-9edad6589f09-maas-api-tls\") pod \"maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.180941 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.180811    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rwhq\" (UniqueName: \"kubernetes.io/projected/55fa83b8-fbae-4d41-a268-9edad6589f09-kube-api-access-9rwhq\") pod \"maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.180941 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.180904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa83b8-fbae-4d41-a268-9edad6589f09-openshift-service-ca\") pod \"maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.181605 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.181539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa83b8-fbae-4d41-a268-9edad6589f09-openshift-service-ca\") pod \"maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.183464 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.183444    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55fa83b8-fbae-4d41-a268-9edad6589f09-maas-api-tls\") pod \"maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.189645 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.189624    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rwhq\" (UniqueName: \"kubernetes.io/projected/55fa83b8-fbae-4d41-a268-9edad6589f09-kube-api-access-9rwhq\") pod \"maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.270305 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.270282    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:00.416702 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.416672    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"]
Jul 27 22:06:00.419987 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:06:00.419949    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55fa83b8_fbae_4d41_a268_9edad6589f09.slice/crio-70d8636b6aa65d5d25ea5a138ec10f2eb56ae3dff88fbc4af98f93a5d6c684be WatchSource:0}: Error finding container 70d8636b6aa65d5d25ea5a138ec10f2eb56ae3dff88fbc4af98f93a5d6c684be: Status 404 returned error can't find the container with id 70d8636b6aa65d5d25ea5a138ec10f2eb56ae3dff88fbc4af98f93a5d6c684be
Jul 27 22:06:00.421318 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.421301    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 22:06:00.744590 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:00.744551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c" event={"ID":"55fa83b8-fbae-4d41-a268-9edad6589f09","Type":"ContainerStarted","Data":"70d8636b6aa65d5d25ea5a138ec10f2eb56ae3dff88fbc4af98f93a5d6c684be"}
Jul 27 22:06:01.750470 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:01.750434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c" event={"ID":"55fa83b8-fbae-4d41-a268-9edad6589f09","Type":"ContainerStarted","Data":"fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a"}
Jul 27 22:06:01.774531 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:01.774482    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c" podStartSLOduration=2.021624356 podStartE2EDuration="2.774467481s" totalImagesPullingTime="752.843125ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:06:01.772906663 +0000 UTC m=+1932.668716115" watchObservedRunningTime="2026-07-27 22:06:01.774467481 +0000 UTC m=+1932.670276934"
Jul 27 22:06:02.755936 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:02.755896    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:08.775017 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:08.774978    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:06:13.811540 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:13.811510    2578 generic.go:356] "Generic (PLEG): container finished" podID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerID="487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18" exitCode=0
Jul 27 22:06:13.811994 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:13.811596    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" event={"ID":"348e3b15-86b9-4162-97b3-b8ef8f035f51","Type":"ContainerDied","Data":"487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18"}
Jul 27 22:06:13.812059 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:13.812040    2578 scope.go:117] "RemoveContainer" containerID="487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18"
Jul 27 22:06:13.813098 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:13.813069    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:06:13.813153 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:13.813142    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:06:13.813206 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:13.813194    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:06:14.738275 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:14.738230    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:06:14.818560 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:14.818517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" event={"ID":"348e3b15-86b9-4162-97b3-b8ef8f035f51","Type":"ContainerStarted","Data":"13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff"}
Jul 27 22:06:14.864399 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:14.864361    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:06:14.865674 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:14.865637    2578 patch_prober.go:28] interesting pod/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 27 22:06:14.865841 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:14.865699    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 27 22:06:15.865055 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:15.864622    2578 patch_prober.go:28] interesting pod/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 27 22:06:15.865055 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:15.864676    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 27 22:06:16.864077 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:16.864037    2578 patch_prober.go:28] interesting pod/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 27 22:06:16.864295 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:16.864122    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 27 22:06:17.864335 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:17.864302    2578 patch_prober.go:28] interesting pod/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 27 22:06:17.864907 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:17.864877    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 27 22:06:18.867263 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:18.867236    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:06:18.867694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:18.867678    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:06:18.868393 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:06:18.868376    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:07:14.680961 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:14.680925    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"]
Jul 27 22:07:14.681495 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:14.681182    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c" podUID="55fa83b8-fbae-4d41-a268-9edad6589f09" containerName="maas-api" containerID="cri-o://fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a" gracePeriod=30
Jul 27 22:07:14.950025 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:14.949954    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:07:15.089814 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.089782    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55fa83b8-fbae-4d41-a268-9edad6589f09-maas-api-tls\") pod \"55fa83b8-fbae-4d41-a268-9edad6589f09\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") "
Jul 27 22:07:15.090020 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.089862    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rwhq\" (UniqueName: \"kubernetes.io/projected/55fa83b8-fbae-4d41-a268-9edad6589f09-kube-api-access-9rwhq\") pod \"55fa83b8-fbae-4d41-a268-9edad6589f09\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") "
Jul 27 22:07:15.090020 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.089905    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa83b8-fbae-4d41-a268-9edad6589f09-openshift-service-ca\") pod \"55fa83b8-fbae-4d41-a268-9edad6589f09\" (UID: \"55fa83b8-fbae-4d41-a268-9edad6589f09\") "
Jul 27 22:07:15.090271 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.090241    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55fa83b8-fbae-4d41-a268-9edad6589f09-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55fa83b8-fbae-4d41-a268-9edad6589f09" (UID: "55fa83b8-fbae-4d41-a268-9edad6589f09"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:07:15.092156 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.092129    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55fa83b8-fbae-4d41-a268-9edad6589f09-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55fa83b8-fbae-4d41-a268-9edad6589f09" (UID: "55fa83b8-fbae-4d41-a268-9edad6589f09"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:07:15.092257 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.092239    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55fa83b8-fbae-4d41-a268-9edad6589f09-kube-api-access-9rwhq" (OuterVolumeSpecName: "kube-api-access-9rwhq") pod "55fa83b8-fbae-4d41-a268-9edad6589f09" (UID: "55fa83b8-fbae-4d41-a268-9edad6589f09"). InnerVolumeSpecName "kube-api-access-9rwhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:07:15.104136 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.104092    2578 generic.go:356] "Generic (PLEG): container finished" podID="55fa83b8-fbae-4d41-a268-9edad6589f09" containerID="fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a" exitCode=0
Jul 27 22:07:15.104231 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.104145    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"
Jul 27 22:07:15.104231 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.104181    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c" event={"ID":"55fa83b8-fbae-4d41-a268-9edad6589f09","Type":"ContainerDied","Data":"fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a"}
Jul 27 22:07:15.104308 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.104232    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c" event={"ID":"55fa83b8-fbae-4d41-a268-9edad6589f09","Type":"ContainerDied","Data":"70d8636b6aa65d5d25ea5a138ec10f2eb56ae3dff88fbc4af98f93a5d6c684be"}
Jul 27 22:07:15.104308 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.104254    2578 scope.go:117] "RemoveContainer" containerID="fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a"
Jul 27 22:07:15.114721 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.114695    2578 scope.go:117] "RemoveContainer" containerID="fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a"
Jul 27 22:07:15.116613 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:07:15.116585    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a\": container with ID starting with fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a not found: ID does not exist" containerID="fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a"
Jul 27 22:07:15.116724 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.116625    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a"} err="failed to get container status \"fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a\": rpc error: code = NotFound desc = could not find container \"fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a\": container with ID starting with fc696eab5fdc57a7eca51ddc6f3887a967ede9222f95b7f36e0a28577c68ec5a not found: ID does not exist"
Jul 27 22:07:15.132086 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.132062    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"]
Jul 27 22:07:15.136020 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.136000    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2733d0b-c7d5f544c-qwj8c"]
Jul 27 22:07:15.190616 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.190582    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55fa83b8-fbae-4d41-a268-9edad6589f09-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:07:15.190616 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.190612    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rwhq\" (UniqueName: \"kubernetes.io/projected/55fa83b8-fbae-4d41-a268-9edad6589f09-kube-api-access-9rwhq\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:07:15.190616 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.190624    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55fa83b8-fbae-4d41-a268-9edad6589f09-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:07:15.605777 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:15.605722    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55fa83b8-fbae-4d41-a268-9edad6589f09" path="/var/lib/kubelet/pods/55fa83b8-fbae-4d41-a268-9edad6589f09/volumes"
Jul 27 22:07:49.703201 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:07:49.703172    2578 scope.go:117] "RemoveContainer" containerID="f0ba3467cf339adbc977aaf33d93d29609a9e116bff7228d5a76c7c9c604b3db"
Jul 27 22:08:06.107137 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:06.107097    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"]
Jul 27 22:08:06.107602 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:06.107310    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy" containerID="cri-o://13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff" gracePeriod=30
Jul 27 22:08:11.360991 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.360967    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:08:11.374972 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.374913    2578 generic.go:356] "Generic (PLEG): container finished" podID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerID="13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff" exitCode=0
Jul 27 22:08:11.374972 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.374956    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"
Jul 27 22:08:11.375098 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.375002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" event={"ID":"348e3b15-86b9-4162-97b3-b8ef8f035f51","Type":"ContainerDied","Data":"13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff"}
Jul 27 22:08:11.375098 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.375056    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj" event={"ID":"348e3b15-86b9-4162-97b3-b8ef8f035f51","Type":"ContainerDied","Data":"ab0bebc84733582e0af859bec8acfe9784eb2e2050055edb9c3e5f3c7c57eb16"}
Jul 27 22:08:11.375098 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.375079    2578 scope.go:117] "RemoveContainer" containerID="13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff"
Jul 27 22:08:11.383216 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.383201    2578 scope.go:117] "RemoveContainer" containerID="487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18"
Jul 27 22:08:11.391599 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.391578    2578 scope.go:117] "RemoveContainer" containerID="13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff"
Jul 27 22:08:11.391929 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:08:11.391904    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff\": container with ID starting with 13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff not found: ID does not exist" containerID="13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff"
Jul 27 22:08:11.392039 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.391939    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff"} err="failed to get container status \"13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff\": rpc error: code = NotFound desc = could not find container \"13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff\": container with ID starting with 13d4955ad23ccd3c019720732c5a1aa6debe9ffbdc7ac41782429f1f307a22ff not found: ID does not exist"
Jul 27 22:08:11.392039 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.391961    2578 scope.go:117] "RemoveContainer" containerID="487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18"
Jul 27 22:08:11.392274 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:08:11.392251    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18\": container with ID starting with 487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18 not found: ID does not exist" containerID="487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18"
Jul 27 22:08:11.392368 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.392283    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18"} err="failed to get container status \"487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18\": rpc error: code = NotFound desc = could not find container \"487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18\": container with ID starting with 487b136e2d942ccecf1c2bdf35e25b4f611dfd61a3d45267351cc817fb979a18 not found: ID does not exist"
Jul 27 22:08:11.429753 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.429711    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-certs\") pod \"348e3b15-86b9-4162-97b3-b8ef8f035f51\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") "
Jul 27 22:08:11.429839 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.429768    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-envoy\") pod \"348e3b15-86b9-4162-97b3-b8ef8f035f51\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") "
Jul 27 22:08:11.429839 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.429793    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-credential-socket\") pod \"348e3b15-86b9-4162-97b3-b8ef8f035f51\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") "
Jul 27 22:08:11.429839 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.429812    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-token\") pod \"348e3b15-86b9-4162-97b3-b8ef8f035f51\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") "
Jul 27 22:08:11.429991 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.429841    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-266nw\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-kube-api-access-266nw\") pod \"348e3b15-86b9-4162-97b3-b8ef8f035f51\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") "
Jul 27 22:08:11.429991 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.429860    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/348e3b15-86b9-4162-97b3-b8ef8f035f51-istiod-ca-cert\") pod \"348e3b15-86b9-4162-97b3-b8ef8f035f51\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") "
Jul 27 22:08:11.429991 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.429898    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-data\") pod \"348e3b15-86b9-4162-97b3-b8ef8f035f51\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") "
Jul 27 22:08:11.429991 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.429946    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-socket\") pod \"348e3b15-86b9-4162-97b3-b8ef8f035f51\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") "
Jul 27 22:08:11.429991 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.429956    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "348e3b15-86b9-4162-97b3-b8ef8f035f51" (UID: "348e3b15-86b9-4162-97b3-b8ef8f035f51"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:08:11.430248 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.430007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-podinfo\") pod \"348e3b15-86b9-4162-97b3-b8ef8f035f51\" (UID: \"348e3b15-86b9-4162-97b3-b8ef8f035f51\") "
Jul 27 22:08:11.430248 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.430059    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "348e3b15-86b9-4162-97b3-b8ef8f035f51" (UID: "348e3b15-86b9-4162-97b3-b8ef8f035f51"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:08:11.430248 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.430211    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "348e3b15-86b9-4162-97b3-b8ef8f035f51" (UID: "348e3b15-86b9-4162-97b3-b8ef8f035f51"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:08:11.430411 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.430265    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/348e3b15-86b9-4162-97b3-b8ef8f035f51-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "348e3b15-86b9-4162-97b3-b8ef8f035f51" (UID: "348e3b15-86b9-4162-97b3-b8ef8f035f51"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:08:11.430411 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.430281    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:08:11.430411 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.430302    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:08:11.431192 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.431167    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-data" (OuterVolumeSpecName: "istio-data") pod "348e3b15-86b9-4162-97b3-b8ef8f035f51" (UID: "348e3b15-86b9-4162-97b3-b8ef8f035f51"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:08:11.432056 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.432031    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "348e3b15-86b9-4162-97b3-b8ef8f035f51" (UID: "348e3b15-86b9-4162-97b3-b8ef8f035f51"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:08:11.432150 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.432064    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-token" (OuterVolumeSpecName: "istio-token") pod "348e3b15-86b9-4162-97b3-b8ef8f035f51" (UID: "348e3b15-86b9-4162-97b3-b8ef8f035f51"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:08:11.432150 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.432129    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-kube-api-access-266nw" (OuterVolumeSpecName: "kube-api-access-266nw") pod "348e3b15-86b9-4162-97b3-b8ef8f035f51" (UID: "348e3b15-86b9-4162-97b3-b8ef8f035f51"). InnerVolumeSpecName "kube-api-access-266nw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:08:11.432389 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.432369    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "348e3b15-86b9-4162-97b3-b8ef8f035f51" (UID: "348e3b15-86b9-4162-97b3-b8ef8f035f51"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:08:11.530941 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.530903    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-266nw\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-kube-api-access-266nw\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:08:11.530941 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.530938    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/348e3b15-86b9-4162-97b3-b8ef8f035f51-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:08:11.531090 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.530952    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:08:11.531090 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.530966    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:08:11.531090 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.530978    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:08:11.531090 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.530990    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:08:11.531090 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.531002    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/348e3b15-86b9-4162-97b3-b8ef8f035f51-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:08:11.692155 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.692129    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"]
Jul 27 22:08:11.697940 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:11.697916    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e2733d0b-openshift-default-7766b6b99b-q7rsj"]
Jul 27 22:08:13.605469 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:13.605431    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" path="/var/lib/kubelet/pods/348e3b15-86b9-4162-97b3-b8ef8f035f51/volumes"
Jul 27 22:08:49.621530 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:49.621502    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:08:49.635711 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:08:49.635684    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:09:01.833795 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.833766    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"]
Jul 27 22:09:01.834334 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.834232    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55fa83b8-fbae-4d41-a268-9edad6589f09" containerName="maas-api"
Jul 27 22:09:01.834334 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.834251    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="55fa83b8-fbae-4d41-a268-9edad6589f09" containerName="maas-api"
Jul 27 22:09:01.834334 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.834278    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy"
Jul 27 22:09:01.834334 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.834287    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy"
Jul 27 22:09:01.834334 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.834300    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy"
Jul 27 22:09:01.834334 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.834308    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy"
Jul 27 22:09:01.834642 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.834393    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy"
Jul 27 22:09:01.834642 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.834409    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="55fa83b8-fbae-4d41-a268-9edad6589f09" containerName="maas-api"
Jul 27 22:09:01.837426 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.837407    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:01.840552 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.840533    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-462b96-openshift-default-dockercfg-4m52c\""
Jul 27 22:09:01.852963 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.852943    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"]
Jul 27 22:09:01.999602 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.999570    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-envoy\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:01.999602 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.999607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-data\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:01.999847 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.999633    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-socket\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:01.999847 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.999654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-certs\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:01.999847 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.999682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crg2j\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-kube-api-access-crg2j\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:01.999847 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.999709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fde132-3c3f-4c45-95dd-eddb649677c0-istiod-ca-cert\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.000043 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.999844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-podinfo\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.000043 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.999895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-token\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.000043 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:01.999926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-credential-socket\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.100939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-envoy\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.100968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-data\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.100995    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-socket\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101290 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101011    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-certs\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101290 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101040    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crg2j\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-kube-api-access-crg2j\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101290 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fde132-3c3f-4c45-95dd-eddb649677c0-istiod-ca-cert\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101290 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-podinfo\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101505 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101373    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-data\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101505 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-token\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101505 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-credential-socket\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101505 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101449    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-socket\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101907 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101546    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-certs\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101907 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101797    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fde132-3c3f-4c45-95dd-eddb649677c0-istiod-ca-cert\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.101907 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.101821    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-credential-socket\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.103465 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.103442    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-envoy\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.103798 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.103775    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-podinfo\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.109894 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.109871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-token\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.110693 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.110671    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crg2j\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-kube-api-access-crg2j\") pod \"e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") " pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.147387 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.147365    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:02.287013 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.286896    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"]
Jul 27 22:09:02.289790 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:09:02.289762    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59fde132_3c3f_4c45_95dd_eddb649677c0.slice/crio-c35ac3ccb078a1acb7e7a2a727c6ad19dd212db3698ced4b4c27628d429e5051 WatchSource:0}: Error finding container c35ac3ccb078a1acb7e7a2a727c6ad19dd212db3698ced4b4c27628d429e5051: Status 404 returned error can't find the container with id c35ac3ccb078a1acb7e7a2a727c6ad19dd212db3698ced4b4c27628d429e5051
Jul 27 22:09:02.292121 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.292084    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:02.292210 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.292152    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:02.292210 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.292184    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:02.623132 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.623096    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" event={"ID":"59fde132-3c3f-4c45-95dd-eddb649677c0","Type":"ContainerStarted","Data":"126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4"}
Jul 27 22:09:02.623306 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.623157    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" event={"ID":"59fde132-3c3f-4c45-95dd-eddb649677c0","Type":"ContainerStarted","Data":"c35ac3ccb078a1acb7e7a2a727c6ad19dd212db3698ced4b4c27628d429e5051"}
Jul 27 22:09:02.646768 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:02.646704    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" podStartSLOduration=1.646689013 podStartE2EDuration="1.646689013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:09:02.645678635 +0000 UTC m=+2113.541488099" watchObservedRunningTime="2026-07-27 22:09:02.646689013 +0000 UTC m=+2113.542498463"
Jul 27 22:09:03.148486 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:03.148458    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:03.153286 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:03.153265    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:03.627349 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:03.627320    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:03.628274 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:03.628255    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:04.718554 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.718521    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"]
Jul 27 22:09:04.719137 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.719119    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="348e3b15-86b9-4162-97b3-b8ef8f035f51" containerName="istio-proxy"
Jul 27 22:09:04.722262 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.722242    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:04.724851 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.724829    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-462b96-serving-cert\""
Jul 27 22:09:04.731844 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.731824    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"]
Jul 27 22:09:04.819101 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.819070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572fa868-4aac-43f4-929d-6040526ab5c1-maas-api-tls\") pod \"maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:04.819250 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.819108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5ztv\" (UniqueName: \"kubernetes.io/projected/572fa868-4aac-43f4-929d-6040526ab5c1-kube-api-access-j5ztv\") pod \"maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:04.819250 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.819136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572fa868-4aac-43f4-929d-6040526ab5c1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:04.920052 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.920028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572fa868-4aac-43f4-929d-6040526ab5c1-maas-api-tls\") pod \"maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:04.920245 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.920062    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5ztv\" (UniqueName: \"kubernetes.io/projected/572fa868-4aac-43f4-929d-6040526ab5c1-kube-api-access-j5ztv\") pod \"maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:04.920245 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.920098    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572fa868-4aac-43f4-929d-6040526ab5c1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:04.920797 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.920770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572fa868-4aac-43f4-929d-6040526ab5c1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:04.922610 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.922588    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572fa868-4aac-43f4-929d-6040526ab5c1-maas-api-tls\") pod \"maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:04.930163 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:04.930142    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5ztv\" (UniqueName: \"kubernetes.io/projected/572fa868-4aac-43f4-929d-6040526ab5c1-kube-api-access-j5ztv\") pod \"maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:05.035100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:05.035047    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:05.378414 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:05.378385    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"]
Jul 27 22:09:05.380419 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:09:05.380385    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod572fa868_4aac_43f4_929d_6040526ab5c1.slice/crio-52dd682de65c7eae0589ec0277fe4c89574963a8b4a30f5f0d28aa8e31285437 WatchSource:0}: Error finding container 52dd682de65c7eae0589ec0277fe4c89574963a8b4a30f5f0d28aa8e31285437: Status 404 returned error can't find the container with id 52dd682de65c7eae0589ec0277fe4c89574963a8b4a30f5f0d28aa8e31285437
Jul 27 22:09:05.637610 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:05.637524    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z" event={"ID":"572fa868-4aac-43f4-929d-6040526ab5c1","Type":"ContainerStarted","Data":"52dd682de65c7eae0589ec0277fe4c89574963a8b4a30f5f0d28aa8e31285437"}
Jul 27 22:09:06.644059 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:06.644018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z" event={"ID":"572fa868-4aac-43f4-929d-6040526ab5c1","Type":"ContainerStarted","Data":"640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4"}
Jul 27 22:09:06.644450 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:06.644151    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:06.662833 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:06.662788    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z" podStartSLOduration=2.187268397 podStartE2EDuration="2.662776067s" totalImagesPullingTime="475.50767ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:09:06.661314366 +0000 UTC m=+2117.557123830" watchObservedRunningTime="2026-07-27 22:09:06.662776067 +0000 UTC m=+2117.558585518"
Jul 27 22:09:12.660721 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:12.660686    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:09:18.629076 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:18.629043    2578 patch_prober.go:28] interesting pod/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 27 22:09:18.629473 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:18.629112    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 27 22:09:18.707241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:18.707213    2578 generic.go:356] "Generic (PLEG): container finished" podID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerID="126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4" exitCode=0
Jul 27 22:09:18.707346 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:18.707257    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" event={"ID":"59fde132-3c3f-4c45-95dd-eddb649677c0","Type":"ContainerDied","Data":"126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4"}
Jul 27 22:09:18.707581 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:18.707564    2578 scope.go:117] "RemoveContainer" containerID="126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4"
Jul 27 22:09:18.708645 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:18.708611    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:18.708728 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:18.708708    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:18.708807 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:18.708791    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:19.714223 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:19.714184    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" event={"ID":"59fde132-3c3f-4c45-95dd-eddb649677c0","Type":"ContainerStarted","Data":"41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5"}
Jul 27 22:09:20.148434 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:20.148333    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:20.148577 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:20.148562    2578 patch_prober.go:28] interesting pod/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 27 22:09:20.148627 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:20.148608    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 27 22:09:21.148428 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:21.148390    2578 patch_prober.go:28] interesting pod/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 27 22:09:21.148851 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:21.148452    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 27 22:09:22.148375 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:22.148340    2578 patch_prober.go:28] interesting pod/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 27 22:09:22.148553 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:22.148395    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 27 22:09:23.147834 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:23.147793    2578 patch_prober.go:28] interesting pod/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 27 22:09:23.148004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:23.147856    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 27 22:09:24.152027 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:24.151998    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:24.152423 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:24.152278    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:24.152887 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:24.152868    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:09:30.405286 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.405252    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"]
Jul 27 22:09:30.408956 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.408933    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.412081 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.412063    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d28642-openshift-default-dockercfg-wnq98\""
Jul 27 22:09:30.424117 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.424078    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"]
Jul 27 22:09:30.532901 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.532865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg5kx\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-kube-api-access-lg5kx\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.533092 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.532926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-certs\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.533092 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.533008    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istiod-ca-cert\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.533092 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.533063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-data\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.533267 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.533089    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-token\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.533267 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.533116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-credential-socket\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.533267 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.533141    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-podinfo\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.533267 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.533184    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-envoy\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.533267 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.533253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-socket\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634338 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-socket\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634338 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg5kx\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-kube-api-access-lg5kx\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634572 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634395    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-certs\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634572 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istiod-ca-cert\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634677 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-data\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634677 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-token\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634677 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-credential-socket\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634894 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-podinfo\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634894 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634747    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-envoy\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.634894 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634781    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-socket\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.635052 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.634998    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-certs\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.635052 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.635030    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-credential-socket\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.635133 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.635059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-data\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.635246 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.635222    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istiod-ca-cert\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.637061 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.637036    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-envoy\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.637257 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.637239    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-podinfo\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.642552 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.642529    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-token\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.642686 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.642668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg5kx\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-kube-api-access-lg5kx\") pod \"e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") " pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.719056 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.718967    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:30.860703 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.860676    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"]
Jul 27 22:09:30.862301 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:09:30.862271    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a40085_bcb8_4a62_bddb_2ac6a17517e3.slice/crio-11b39741eb436e77f3e4400f9a42e6407c092e638393349014d978002996885a WatchSource:0}: Error finding container 11b39741eb436e77f3e4400f9a42e6407c092e638393349014d978002996885a: Status 404 returned error can't find the container with id 11b39741eb436e77f3e4400f9a42e6407c092e638393349014d978002996885a
Jul 27 22:09:30.864388 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.864355    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:30.864493 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.864416    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:30.864493 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:30.864462    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:31.773964 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:31.773927    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" event={"ID":"60a40085-bcb8-4a62-bddb-2ac6a17517e3","Type":"ContainerStarted","Data":"5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0"}
Jul 27 22:09:31.774350 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:31.773996    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" event={"ID":"60a40085-bcb8-4a62-bddb-2ac6a17517e3","Type":"ContainerStarted","Data":"11b39741eb436e77f3e4400f9a42e6407c092e638393349014d978002996885a"}
Jul 27 22:09:31.799262 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:31.799215    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" podStartSLOduration=1.799200371 podStartE2EDuration="1.799200371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:09:31.797125448 +0000 UTC m=+2142.692934901" watchObservedRunningTime="2026-07-27 22:09:31.799200371 +0000 UTC m=+2142.695009823"
Jul 27 22:09:32.719985 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:32.719952    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:32.724760 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:32.724712    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:32.779260 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:32.779231    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:32.780158 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:32.780139    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:33.812933 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.812902    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"]
Jul 27 22:09:33.817275 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.817253    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:33.819982 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.819959    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d28642-serving-cert\""
Jul 27 22:09:33.825201 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.825179    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"]
Jul 27 22:09:33.864148 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.864120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d634d748-645c-40b5-bac1-20270f0ecd12-maas-api-tls\") pod \"maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:33.864270 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.864201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfhxg\" (UniqueName: \"kubernetes.io/projected/d634d748-645c-40b5-bac1-20270f0ecd12-kube-api-access-xfhxg\") pod \"maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:33.864334 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.864303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d634d748-645c-40b5-bac1-20270f0ecd12-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:33.965121 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.965087    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d634d748-645c-40b5-bac1-20270f0ecd12-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:33.965323 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.965132    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d634d748-645c-40b5-bac1-20270f0ecd12-maas-api-tls\") pod \"maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:33.965323 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.965164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfhxg\" (UniqueName: \"kubernetes.io/projected/d634d748-645c-40b5-bac1-20270f0ecd12-kube-api-access-xfhxg\") pod \"maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:33.965839 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.965814    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d634d748-645c-40b5-bac1-20270f0ecd12-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:33.967934 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.967911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d634d748-645c-40b5-bac1-20270f0ecd12-maas-api-tls\") pod \"maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:33.975537 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:33.975515    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfhxg\" (UniqueName: \"kubernetes.io/projected/d634d748-645c-40b5-bac1-20270f0ecd12-kube-api-access-xfhxg\") pod \"maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:34.130879 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:34.130815    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:34.272383 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:34.272360    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"]
Jul 27 22:09:34.274821 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:09:34.274794    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd634d748_645c_40b5_bac1_20270f0ecd12.slice/crio-5b22655feffd3c6a7dfc9f315451b72a558fbaa616d84d2a8b22fea71b931b07 WatchSource:0}: Error finding container 5b22655feffd3c6a7dfc9f315451b72a558fbaa616d84d2a8b22fea71b931b07: Status 404 returned error can't find the container with id 5b22655feffd3c6a7dfc9f315451b72a558fbaa616d84d2a8b22fea71b931b07
Jul 27 22:09:34.790185 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:34.790153    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8" event={"ID":"d634d748-645c-40b5-bac1-20270f0ecd12","Type":"ContainerStarted","Data":"6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83"}
Jul 27 22:09:34.790292 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:34.790236    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8" event={"ID":"d634d748-645c-40b5-bac1-20270f0ecd12","Type":"ContainerStarted","Data":"5b22655feffd3c6a7dfc9f315451b72a558fbaa616d84d2a8b22fea71b931b07"}
Jul 27 22:09:34.790424 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:34.790408    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:34.807987 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:34.807946    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8" podStartSLOduration=1.384356897 podStartE2EDuration="1.807932596s" totalImagesPullingTime="423.575699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:09:34.807031304 +0000 UTC m=+2145.702840780" watchObservedRunningTime="2026-07-27 22:09:34.807932596 +0000 UTC m=+2145.703742046"
Jul 27 22:09:40.800524 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:40.800450    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:09:48.819946 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:48.819905    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 27 22:09:48.820321 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:48.820018    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 27 22:09:49.734320 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:49.734249    2578 scope.go:117] "RemoveContainer" containerID="f2c835ddcdea50adcb8017e9ccd3b0b2eda09e3c6f9c6b365ac55fbe9e07446d"
Jul 27 22:09:49.871239 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:49.871152    2578 generic.go:356] "Generic (PLEG): container finished" podID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerID="5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0" exitCode=0
Jul 27 22:09:49.871239 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:49.871203    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" event={"ID":"60a40085-bcb8-4a62-bddb-2ac6a17517e3","Type":"ContainerDied","Data":"5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0"}
Jul 27 22:09:49.871725 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:49.871650    2578 scope.go:117] "RemoveContainer" containerID="5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0"
Jul 27 22:09:49.872702 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:49.872674    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:49.872797 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:49.872755    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:49.872797 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:49.872789    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:09:50.880510 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:50.880474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" event={"ID":"60a40085-bcb8-4a62-bddb-2ac6a17517e3","Type":"ContainerStarted","Data":"4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601"}
Jul 27 22:09:51.719790 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:51.719754    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:51.719989 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:51.719967    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 27 22:09:51.720096 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:51.720004    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 27 22:09:52.719899 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:52.719866    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 27 22:09:52.720287 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:52.719917    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 27 22:09:53.719718 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:53.719677    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 27 22:09:53.719896 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:53.719761    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 27 22:09:54.719718 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:54.719686    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 27 22:09:54.720091 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:54.719763    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 27 22:09:55.723301 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:55.723274    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:55.723678 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:55.723533    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:09:55.724155 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:09:55.724139    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:11:35.119008 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.118978    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"]
Jul 27 22:11:35.119505 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.119189    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z" podUID="572fa868-4aac-43f4-929d-6040526ab5c1" containerName="maas-api" containerID="cri-o://640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4" gracePeriod=30
Jul 27 22:11:35.384335 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.384268    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:11:35.395722 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.395694    2578 generic.go:356] "Generic (PLEG): container finished" podID="572fa868-4aac-43f4-929d-6040526ab5c1" containerID="640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4" exitCode=0
Jul 27 22:11:35.395952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.395801    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"
Jul 27 22:11:35.395952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.395841    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z" event={"ID":"572fa868-4aac-43f4-929d-6040526ab5c1","Type":"ContainerDied","Data":"640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4"}
Jul 27 22:11:35.395952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.395889    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z" event={"ID":"572fa868-4aac-43f4-929d-6040526ab5c1","Type":"ContainerDied","Data":"52dd682de65c7eae0589ec0277fe4c89574963a8b4a30f5f0d28aa8e31285437"}
Jul 27 22:11:35.395952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.395911    2578 scope.go:117] "RemoveContainer" containerID="640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4"
Jul 27 22:11:35.404905 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.404886    2578 scope.go:117] "RemoveContainer" containerID="640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4"
Jul 27 22:11:35.405206 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:11:35.405181    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4\": container with ID starting with 640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4 not found: ID does not exist" containerID="640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4"
Jul 27 22:11:35.405356 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.405218    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4"} err="failed to get container status \"640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4\": rpc error: code = NotFound desc = could not find container \"640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4\": container with ID starting with 640729242ea0527e0e997136c542b30512bb200400b378e34d5d0df1f22601a4 not found: ID does not exist"
Jul 27 22:11:35.470823 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.470797    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572fa868-4aac-43f4-929d-6040526ab5c1-maas-api-tls\") pod \"572fa868-4aac-43f4-929d-6040526ab5c1\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") "
Jul 27 22:11:35.470968 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.470889    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5ztv\" (UniqueName: \"kubernetes.io/projected/572fa868-4aac-43f4-929d-6040526ab5c1-kube-api-access-j5ztv\") pod \"572fa868-4aac-43f4-929d-6040526ab5c1\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") "
Jul 27 22:11:35.470968 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.470948    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572fa868-4aac-43f4-929d-6040526ab5c1-openshift-service-ca\") pod \"572fa868-4aac-43f4-929d-6040526ab5c1\" (UID: \"572fa868-4aac-43f4-929d-6040526ab5c1\") "
Jul 27 22:11:35.471715 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.471673    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572fa868-4aac-43f4-929d-6040526ab5c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "572fa868-4aac-43f4-929d-6040526ab5c1" (UID: "572fa868-4aac-43f4-929d-6040526ab5c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:11:35.473543 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.473511    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572fa868-4aac-43f4-929d-6040526ab5c1-kube-api-access-j5ztv" (OuterVolumeSpecName: "kube-api-access-j5ztv") pod "572fa868-4aac-43f4-929d-6040526ab5c1" (UID: "572fa868-4aac-43f4-929d-6040526ab5c1"). InnerVolumeSpecName "kube-api-access-j5ztv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:11:35.473543 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.473517    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/572fa868-4aac-43f4-929d-6040526ab5c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "572fa868-4aac-43f4-929d-6040526ab5c1" (UID: "572fa868-4aac-43f4-929d-6040526ab5c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:11:35.571888 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.571856    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572fa868-4aac-43f4-929d-6040526ab5c1-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:35.571888 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.571885    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572fa868-4aac-43f4-929d-6040526ab5c1-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:35.571888 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.571895    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5ztv\" (UniqueName: \"kubernetes.io/projected/572fa868-4aac-43f4-929d-6040526ab5c1-kube-api-access-j5ztv\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:35.724307 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.724275    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"]
Jul 27 22:11:35.731017 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:35.730987    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-462b96-845fb847d8-4vs2z"]
Jul 27 22:11:37.604797 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:37.604756    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="572fa868-4aac-43f4-929d-6040526ab5c1" path="/var/lib/kubelet/pods/572fa868-4aac-43f4-929d-6040526ab5c1/volumes"
Jul 27 22:11:41.670044 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:41.670010    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"]
Jul 27 22:11:41.670481 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:41.670207    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy" containerID="cri-o://41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5" gracePeriod=30
Jul 27 22:11:46.924696 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:46.924673    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:11:47.068905 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.068826    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-envoy\") pod \"59fde132-3c3f-4c45-95dd-eddb649677c0\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") "
Jul 27 22:11:47.068905 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.068856    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-socket\") pod \"59fde132-3c3f-4c45-95dd-eddb649677c0\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") "
Jul 27 22:11:47.068905 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.068886    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fde132-3c3f-4c45-95dd-eddb649677c0-istiod-ca-cert\") pod \"59fde132-3c3f-4c45-95dd-eddb649677c0\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") "
Jul 27 22:11:47.069199 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.068914    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-certs\") pod \"59fde132-3c3f-4c45-95dd-eddb649677c0\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") "
Jul 27 22:11:47.069199 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.068944    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-token\") pod \"59fde132-3c3f-4c45-95dd-eddb649677c0\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") "
Jul 27 22:11:47.069199 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.068974    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-podinfo\") pod \"59fde132-3c3f-4c45-95dd-eddb649677c0\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") "
Jul 27 22:11:47.069199 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.069013    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-credential-socket\") pod \"59fde132-3c3f-4c45-95dd-eddb649677c0\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") "
Jul 27 22:11:47.069199 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.069067    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-data\") pod \"59fde132-3c3f-4c45-95dd-eddb649677c0\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") "
Jul 27 22:11:47.069199 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.069098    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crg2j\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-kube-api-access-crg2j\") pod \"59fde132-3c3f-4c45-95dd-eddb649677c0\" (UID: \"59fde132-3c3f-4c45-95dd-eddb649677c0\") "
Jul 27 22:11:47.069199 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.069163    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59fde132-3c3f-4c45-95dd-eddb649677c0" (UID: "59fde132-3c3f-4c45-95dd-eddb649677c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:11:47.069512 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.069375    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59fde132-3c3f-4c45-95dd-eddb649677c0" (UID: "59fde132-3c3f-4c45-95dd-eddb649677c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:11:47.069512 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.069389    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59fde132-3c3f-4c45-95dd-eddb649677c0" (UID: "59fde132-3c3f-4c45-95dd-eddb649677c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:11:47.069512 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.069390    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59fde132-3c3f-4c45-95dd-eddb649677c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59fde132-3c3f-4c45-95dd-eddb649677c0" (UID: "59fde132-3c3f-4c45-95dd-eddb649677c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:11:47.069512 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.069431    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:47.070359 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.070335    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "59fde132-3c3f-4c45-95dd-eddb649677c0" (UID: "59fde132-3c3f-4c45-95dd-eddb649677c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:11:47.071423 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.071403    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-kube-api-access-crg2j" (OuterVolumeSpecName: "kube-api-access-crg2j") pod "59fde132-3c3f-4c45-95dd-eddb649677c0" (UID: "59fde132-3c3f-4c45-95dd-eddb649677c0"). InnerVolumeSpecName "kube-api-access-crg2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:11:47.071423 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.071413    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59fde132-3c3f-4c45-95dd-eddb649677c0" (UID: "59fde132-3c3f-4c45-95dd-eddb649677c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:11:47.071876 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.071858    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59fde132-3c3f-4c45-95dd-eddb649677c0" (UID: "59fde132-3c3f-4c45-95dd-eddb649677c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:11:47.071876 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.071863    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "59fde132-3c3f-4c45-95dd-eddb649677c0" (UID: "59fde132-3c3f-4c45-95dd-eddb649677c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:11:47.170930 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.170900    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:47.170930 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.170927    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crg2j\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-kube-api-access-crg2j\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:47.170930 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.170936    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:47.171155 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.170945    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59fde132-3c3f-4c45-95dd-eddb649677c0-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:47.171155 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.170953    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:47.171155 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.170960    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:47.171155 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.170967    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59fde132-3c3f-4c45-95dd-eddb649677c0-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:47.171155 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.170975    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59fde132-3c3f-4c45-95dd-eddb649677c0-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:47.453872 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.453841    2578 generic.go:356] "Generic (PLEG): container finished" podID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerID="41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5" exitCode=0
Jul 27 22:11:47.454078 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.453904    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"
Jul 27 22:11:47.454078 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.453935    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" event={"ID":"59fde132-3c3f-4c45-95dd-eddb649677c0","Type":"ContainerDied","Data":"41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5"}
Jul 27 22:11:47.454078 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.454003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb" event={"ID":"59fde132-3c3f-4c45-95dd-eddb649677c0","Type":"ContainerDied","Data":"c35ac3ccb078a1acb7e7a2a727c6ad19dd212db3698ced4b4c27628d429e5051"}
Jul 27 22:11:47.454078 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.454038    2578 scope.go:117] "RemoveContainer" containerID="41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5"
Jul 27 22:11:47.464305 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.464289    2578 scope.go:117] "RemoveContainer" containerID="126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4"
Jul 27 22:11:47.472421 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.472405    2578 scope.go:117] "RemoveContainer" containerID="41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5"
Jul 27 22:11:47.472679 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:11:47.472655    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5\": container with ID starting with 41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5 not found: ID does not exist" containerID="41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5"
Jul 27 22:11:47.472768 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.472691    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5"} err="failed to get container status \"41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5\": rpc error: code = NotFound desc = could not find container \"41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5\": container with ID starting with 41a83e68bb2ff9305d418a17ec430c5216847a5cc9bdca64c420dc4ac7e722f5 not found: ID does not exist"
Jul 27 22:11:47.472768 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.472715    2578 scope.go:117] "RemoveContainer" containerID="126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4"
Jul 27 22:11:47.473003 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:11:47.472985    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4\": container with ID starting with 126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4 not found: ID does not exist" containerID="126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4"
Jul 27 22:11:47.473060 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.473006    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4"} err="failed to get container status \"126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4\": rpc error: code = NotFound desc = could not find container \"126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4\": container with ID starting with 126e9f0216297787955bdf26d383ff88f2d4aed0175e37efa7bdd4a2774cd5d4 not found: ID does not exist"
Jul 27 22:11:47.478327 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.478304    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"]
Jul 27 22:11:47.484170 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.484149    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-462b96-openshift-default-6f584fc75b-497nb"]
Jul 27 22:11:47.605634 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:47.605605    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" path="/var/lib/kubelet/pods/59fde132-3c3f-4c45-95dd-eddb649677c0/volumes"
Jul 27 22:11:52.425800 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:52.425766    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"]
Jul 27 22:11:52.426584 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:52.426548    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8" podUID="d634d748-645c-40b5-bac1-20270f0ecd12" containerName="maas-api" containerID="cri-o://6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83" gracePeriod=30
Jul 27 22:11:52.887317 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:52.887295    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:11:53.017061 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.016995    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d634d748-645c-40b5-bac1-20270f0ecd12-maas-api-tls\") pod \"d634d748-645c-40b5-bac1-20270f0ecd12\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") "
Jul 27 22:11:53.017061 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.017027    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfhxg\" (UniqueName: \"kubernetes.io/projected/d634d748-645c-40b5-bac1-20270f0ecd12-kube-api-access-xfhxg\") pod \"d634d748-645c-40b5-bac1-20270f0ecd12\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") "
Jul 27 22:11:53.017061 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.017048    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d634d748-645c-40b5-bac1-20270f0ecd12-openshift-service-ca\") pod \"d634d748-645c-40b5-bac1-20270f0ecd12\" (UID: \"d634d748-645c-40b5-bac1-20270f0ecd12\") "
Jul 27 22:11:53.017398 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.017375    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d634d748-645c-40b5-bac1-20270f0ecd12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d634d748-645c-40b5-bac1-20270f0ecd12" (UID: "d634d748-645c-40b5-bac1-20270f0ecd12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:11:53.019257 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.019229    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d634d748-645c-40b5-bac1-20270f0ecd12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d634d748-645c-40b5-bac1-20270f0ecd12" (UID: "d634d748-645c-40b5-bac1-20270f0ecd12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:11:53.019346 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.019262    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d634d748-645c-40b5-bac1-20270f0ecd12-kube-api-access-xfhxg" (OuterVolumeSpecName: "kube-api-access-xfhxg") pod "d634d748-645c-40b5-bac1-20270f0ecd12" (UID: "d634d748-645c-40b5-bac1-20270f0ecd12"). InnerVolumeSpecName "kube-api-access-xfhxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:11:53.117948 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.117924    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d634d748-645c-40b5-bac1-20270f0ecd12-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:53.117948 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.117945    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfhxg\" (UniqueName: \"kubernetes.io/projected/d634d748-645c-40b5-bac1-20270f0ecd12-kube-api-access-xfhxg\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:53.118104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.117956    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d634d748-645c-40b5-bac1-20270f0ecd12-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:11:53.485058 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.485030    2578 generic.go:356] "Generic (PLEG): container finished" podID="d634d748-645c-40b5-bac1-20270f0ecd12" containerID="6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83" exitCode=0
Jul 27 22:11:53.485396 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.485077    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"
Jul 27 22:11:53.485396 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.485121    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8" event={"ID":"d634d748-645c-40b5-bac1-20270f0ecd12","Type":"ContainerDied","Data":"6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83"}
Jul 27 22:11:53.485396 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.485168    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8" event={"ID":"d634d748-645c-40b5-bac1-20270f0ecd12","Type":"ContainerDied","Data":"5b22655feffd3c6a7dfc9f315451b72a558fbaa616d84d2a8b22fea71b931b07"}
Jul 27 22:11:53.485396 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.485189    2578 scope.go:117] "RemoveContainer" containerID="6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83"
Jul 27 22:11:53.494059 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.494042    2578 scope.go:117] "RemoveContainer" containerID="6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83"
Jul 27 22:11:53.494320 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:11:53.494302    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83\": container with ID starting with 6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83 not found: ID does not exist" containerID="6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83"
Jul 27 22:11:53.494380 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.494331    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83"} err="failed to get container status \"6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83\": rpc error: code = NotFound desc = could not find container \"6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83\": container with ID starting with 6abbed6f0a26dd4a32ce9ea0556cc7f54140a4fc1d12b14a5564e265ee748b83 not found: ID does not exist"
Jul 27 22:11:53.510711 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.510685    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"]
Jul 27 22:11:53.514202 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.514184    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d28642-5dbfd6cb8d-d4vl8"]
Jul 27 22:11:53.605205 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:53.605183    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d634d748-645c-40b5-bac1-20270f0ecd12" path="/var/lib/kubelet/pods/d634d748-645c-40b5-bac1-20270f0ecd12/volumes"
Jul 27 22:11:59.254196 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:59.254161    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"]
Jul 27 22:11:59.254621 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:11:59.254375    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy" containerID="cri-o://4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601" gracePeriod=30
Jul 27 22:12:00.164767 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.164714    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"]
Jul 27 22:12:00.165094 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.165080    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="572fa868-4aac-43f4-929d-6040526ab5c1" containerName="maas-api"
Jul 27 22:12:00.165140 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.165097    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="572fa868-4aac-43f4-929d-6040526ab5c1" containerName="maas-api"
Jul 27 22:12:00.165140 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.165109    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d634d748-645c-40b5-bac1-20270f0ecd12" containerName="maas-api"
Jul 27 22:12:00.165140 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.165115    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d634d748-645c-40b5-bac1-20270f0ecd12" containerName="maas-api"
Jul 27 22:12:00.165140 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.165128    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy"
Jul 27 22:12:00.165140 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.165133    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy"
Jul 27 22:12:00.165293 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.165193    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d634d748-645c-40b5-bac1-20270f0ecd12" containerName="maas-api"
Jul 27 22:12:00.165293 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.165201    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="572fa868-4aac-43f4-929d-6040526ab5c1" containerName="maas-api"
Jul 27 22:12:00.165293 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.165206    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy"
Jul 27 22:12:00.169376 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.169358    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.172592 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.172569    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2f19c4-openshift-default-dockercfg-jzztf\""
Jul 27 22:12:00.181057 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.181036    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"]
Jul 27 22:12:00.275611 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.275580    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-credential-socket\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.276047 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.275618    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-certs\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.276047 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.275651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istiod-ca-cert\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.276047 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.275682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-token\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.276047 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.275709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-envoy\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.276047 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.275724    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-podinfo\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.276047 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.275862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-socket\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.276047 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.275906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvhmp\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-kube-api-access-nvhmp\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.276047 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.275948    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-data\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376282 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376260    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvhmp\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-kube-api-access-nvhmp\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376420 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376295    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-data\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376420 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-credential-socket\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376420 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-certs\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376420 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376381    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istiod-ca-cert\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376420 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376398    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-token\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-envoy\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-podinfo\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-socket\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376920 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-data\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376920 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-certs\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.376920 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376904    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-socket\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.377039 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.376998    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-credential-socket\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.377244 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.377225    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istiod-ca-cert\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.379100 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.379074    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-envoy\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.379403 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.379383    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-podinfo\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.392524 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.392505    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-token\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.394444 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.394427    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvhmp\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-kube-api-access-nvhmp\") pod \"e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") " pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.480244 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.480187    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:00.617684 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.617656    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"]
Jul 27 22:12:00.619521 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:12:00.619497    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056dda74_35a2_44fd_9e6e_2d4cb020dd9a.slice/crio-fda48e95f484e13bbce32fcd21941bc9c028864c8d64f5fc63ca8bfe088c4f9a WatchSource:0}: Error finding container fda48e95f484e13bbce32fcd21941bc9c028864c8d64f5fc63ca8bfe088c4f9a: Status 404 returned error can't find the container with id fda48e95f484e13bbce32fcd21941bc9c028864c8d64f5fc63ca8bfe088c4f9a
Jul 27 22:12:00.621562 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.621531    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:12:00.621623 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.621596    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:12:00.621666 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:00.621621    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:12:01.526696 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:01.526662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" event={"ID":"056dda74-35a2-44fd-9e6e-2d4cb020dd9a","Type":"ContainerStarted","Data":"bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639"}
Jul 27 22:12:01.527071 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:01.526763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" event={"ID":"056dda74-35a2-44fd-9e6e-2d4cb020dd9a","Type":"ContainerStarted","Data":"fda48e95f484e13bbce32fcd21941bc9c028864c8d64f5fc63ca8bfe088c4f9a"}
Jul 27 22:12:01.550531 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:01.550485    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" podStartSLOduration=1.550471062 podStartE2EDuration="1.550471062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:12:01.54832278 +0000 UTC m=+2292.444132232" watchObservedRunningTime="2026-07-27 22:12:01.550471062 +0000 UTC m=+2292.446280513"
Jul 27 22:12:02.480541 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:02.480509    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:02.485479 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:02.485455    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:02.531593 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:02.531567    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:02.532441 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:02.532423    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:03.569450 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.569420    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"]
Jul 27 22:12:03.569828 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.569800    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy"
Jul 27 22:12:03.569828 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.569814    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy"
Jul 27 22:12:03.569909 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.569878    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fde132-3c3f-4c45-95dd-eddb649677c0" containerName="istio-proxy"
Jul 27 22:12:03.573259 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.573239    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:03.575716 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.575696    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-2f19c4-serving-cert\""
Jul 27 22:12:03.584491 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.584466    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"]
Jul 27 22:12:03.705437 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.705398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d595fe85-07dd-46f2-b258-c8e1e0850caa-openshift-service-ca\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:03.705628 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.705479    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d595fe85-07dd-46f2-b258-c8e1e0850caa-maas-api-tls\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:03.705628 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.705509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwfqz\" (UniqueName: \"kubernetes.io/projected/d595fe85-07dd-46f2-b258-c8e1e0850caa-kube-api-access-lwfqz\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:03.806185 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.806150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d595fe85-07dd-46f2-b258-c8e1e0850caa-openshift-service-ca\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:03.806338 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.806209    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d595fe85-07dd-46f2-b258-c8e1e0850caa-maas-api-tls\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:03.806338 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.806238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwfqz\" (UniqueName: \"kubernetes.io/projected/d595fe85-07dd-46f2-b258-c8e1e0850caa-kube-api-access-lwfqz\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:03.806338 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:12:03.806299    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-serving-cert: secret "maas-api-e2e-api-a-2f19c4-serving-cert" not found
Jul 27 22:12:03.806493 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:12:03.806363    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d595fe85-07dd-46f2-b258-c8e1e0850caa-maas-api-tls podName:d595fe85-07dd-46f2-b258-c8e1e0850caa nodeName:}" failed. No retries permitted until 2026-07-27 22:12:04.306345945 +0000 UTC m=+2295.202155374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d595fe85-07dd-46f2-b258-c8e1e0850caa-maas-api-tls") pod "maas-api-e2e-api-a-2f19c4-7959965b57-n254d" (UID: "d595fe85-07dd-46f2-b258-c8e1e0850caa") : secret "maas-api-e2e-api-a-2f19c4-serving-cert" not found
Jul 27 22:12:03.806766 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.806727    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d595fe85-07dd-46f2-b258-c8e1e0850caa-openshift-service-ca\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:03.816546 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:03.816525    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwfqz\" (UniqueName: \"kubernetes.io/projected/d595fe85-07dd-46f2-b258-c8e1e0850caa-kube-api-access-lwfqz\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:04.309659 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.309632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d595fe85-07dd-46f2-b258-c8e1e0850caa-maas-api-tls\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:04.312189 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.312150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d595fe85-07dd-46f2-b258-c8e1e0850caa-maas-api-tls\") pod \"maas-api-e2e-api-a-2f19c4-7959965b57-n254d\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:04.484916 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.484889    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:04.520481 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.520426    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:12:04.548456 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.547312    2578 generic.go:356] "Generic (PLEG): container finished" podID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerID="4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601" exitCode=0
Jul 27 22:12:04.548456 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.547374    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"
Jul 27 22:12:04.548456 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.547399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" event={"ID":"60a40085-bcb8-4a62-bddb-2ac6a17517e3","Type":"ContainerDied","Data":"4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601"}
Jul 27 22:12:04.548456 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.547462    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd" event={"ID":"60a40085-bcb8-4a62-bddb-2ac6a17517e3","Type":"ContainerDied","Data":"11b39741eb436e77f3e4400f9a42e6407c092e638393349014d978002996885a"}
Jul 27 22:12:04.548456 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.547485    2578 scope.go:117] "RemoveContainer" containerID="4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601"
Jul 27 22:12:04.560886 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.560801    2578 scope.go:117] "RemoveContainer" containerID="5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0"
Jul 27 22:12:04.571619 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.571589    2578 scope.go:117] "RemoveContainer" containerID="4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601"
Jul 27 22:12:04.571998 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:12:04.571960    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601\": container with ID starting with 4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601 not found: ID does not exist" containerID="4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601"
Jul 27 22:12:04.571998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.571989    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601"} err="failed to get container status \"4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601\": rpc error: code = NotFound desc = could not find container \"4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601\": container with ID starting with 4a7bf2e566a32af19c8c72f68ed0ce44c305f20349f46ab605fe6a9083bf9601 not found: ID does not exist"
Jul 27 22:12:04.572138 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.572009    2578 scope.go:117] "RemoveContainer" containerID="5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0"
Jul 27 22:12:04.572325 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:12:04.572302    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0\": container with ID starting with 5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0 not found: ID does not exist" containerID="5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0"
Jul 27 22:12:04.572429 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.572327    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0"} err="failed to get container status \"5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0\": rpc error: code = NotFound desc = could not find container \"5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0\": container with ID starting with 5ba3095ecaaa89522a05d5c825d9fe1ee90fa57b9ac3920c281636772ea628b0 not found: ID does not exist"
Jul 27 22:12:04.634279 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.634259    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"]
Jul 27 22:12:04.636306 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:12:04.636283    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd595fe85_07dd_46f2_b258_c8e1e0850caa.slice/crio-9fd6b38e3908ee9c0215a44962b0feb53876ae72b3fb95b8f4576e0d5e9f74ff WatchSource:0}: Error finding container 9fd6b38e3908ee9c0215a44962b0feb53876ae72b3fb95b8f4576e0d5e9f74ff: Status 404 returned error can't find the container with id 9fd6b38e3908ee9c0215a44962b0feb53876ae72b3fb95b8f4576e0d5e9f74ff
Jul 27 22:12:04.637718 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.637702    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 22:12:04.713621 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.713594    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-envoy\") pod \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") "
Jul 27 22:12:04.713724 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.713647    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-credential-socket\") pod \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") "
Jul 27 22:12:04.713724 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.713712    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg5kx\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-kube-api-access-lg5kx\") pod \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") "
Jul 27 22:12:04.713850 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.713761    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-data\") pod \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") "
Jul 27 22:12:04.713850 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.713785    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-socket\") pod \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") "
Jul 27 22:12:04.713850 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.713811    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-certs\") pod \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") "
Jul 27 22:12:04.714004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.713832    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istiod-ca-cert\") pod \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") "
Jul 27 22:12:04.714004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.713949    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60a40085-bcb8-4a62-bddb-2ac6a17517e3" (UID: "60a40085-bcb8-4a62-bddb-2ac6a17517e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:12:04.714104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714047    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-podinfo\") pod \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") "
Jul 27 22:12:04.714104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714051    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60a40085-bcb8-4a62-bddb-2ac6a17517e3" (UID: "60a40085-bcb8-4a62-bddb-2ac6a17517e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:12:04.714104 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714068    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60a40085-bcb8-4a62-bddb-2ac6a17517e3" (UID: "60a40085-bcb8-4a62-bddb-2ac6a17517e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:12:04.714235 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714204    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60a40085-bcb8-4a62-bddb-2ac6a17517e3" (UID: "60a40085-bcb8-4a62-bddb-2ac6a17517e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:12:04.714285 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714078    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-token\") pod \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\" (UID: \"60a40085-bcb8-4a62-bddb-2ac6a17517e3\") "
Jul 27 22:12:04.714674 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714566    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:12:04.714674 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714591    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:12:04.714674 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714605    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:12:04.714674 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714621    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:12:04.714674 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.714689    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-data" (OuterVolumeSpecName: "istio-data") pod "60a40085-bcb8-4a62-bddb-2ac6a17517e3" (UID: "60a40085-bcb8-4a62-bddb-2ac6a17517e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:12:04.716198 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.716170    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60a40085-bcb8-4a62-bddb-2ac6a17517e3" (UID: "60a40085-bcb8-4a62-bddb-2ac6a17517e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:12:04.716287 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.716252    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "60a40085-bcb8-4a62-bddb-2ac6a17517e3" (UID: "60a40085-bcb8-4a62-bddb-2ac6a17517e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:12:04.716422 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.716406    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60a40085-bcb8-4a62-bddb-2ac6a17517e3" (UID: "60a40085-bcb8-4a62-bddb-2ac6a17517e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:12:04.716481 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.716434    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-kube-api-access-lg5kx" (OuterVolumeSpecName: "kube-api-access-lg5kx") pod "60a40085-bcb8-4a62-bddb-2ac6a17517e3" (UID: "60a40085-bcb8-4a62-bddb-2ac6a17517e3"). InnerVolumeSpecName "kube-api-access-lg5kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:12:04.815864 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.815801    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:12:04.815864 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.815825    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:12:04.815864 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.815837    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:12:04.815864 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.815849    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg5kx\" (UniqueName: \"kubernetes.io/projected/60a40085-bcb8-4a62-bddb-2ac6a17517e3-kube-api-access-lg5kx\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:12:04.815864 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.815863    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a40085-bcb8-4a62-bddb-2ac6a17517e3-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:12:04.870906 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.870880    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"]
Jul 27 22:12:04.874228 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:04.874204    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d28642-openshift-default-658f5f76d8-94chd"]
Jul 27 22:12:05.554844 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:05.554815    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d" event={"ID":"d595fe85-07dd-46f2-b258-c8e1e0850caa","Type":"ContainerStarted","Data":"5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91"}
Jul 27 22:12:05.554979 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:05.554862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d" event={"ID":"d595fe85-07dd-46f2-b258-c8e1e0850caa","Type":"ContainerStarted","Data":"9fd6b38e3908ee9c0215a44962b0feb53876ae72b3fb95b8f4576e0d5e9f74ff"}
Jul 27 22:12:05.554979 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:05.554892    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:05.573370 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:05.573325    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d" podStartSLOduration=2.120095981 podStartE2EDuration="2.573310265s" totalImagesPullingTime="453.214284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:12:05.571509218 +0000 UTC m=+2296.467318681" watchObservedRunningTime="2026-07-27 22:12:05.573310265 +0000 UTC m=+2296.469119717"
Jul 27 22:12:05.612202 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:05.612173    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" path="/var/lib/kubelet/pods/60a40085-bcb8-4a62-bddb-2ac6a17517e3/volumes"
Jul 27 22:12:11.575074 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:11.575037    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:12:16.613345 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:16.613316    2578 generic.go:356] "Generic (PLEG): container finished" podID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerID="bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639" exitCode=0
Jul 27 22:12:16.613795 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:16.613387    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" event={"ID":"056dda74-35a2-44fd-9e6e-2d4cb020dd9a","Type":"ContainerDied","Data":"bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639"}
Jul 27 22:12:16.613795 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:16.613708    2578 scope.go:117] "RemoveContainer" containerID="bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639"
Jul 27 22:12:16.614787 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:16.614759    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:12:16.614859 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:16.614831    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:12:16.614897 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:16.614861    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:12:17.532195 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:17.532163    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:17.619553 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:17.619517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" event={"ID":"056dda74-35a2-44fd-9e6e-2d4cb020dd9a","Type":"ContainerStarted","Data":"672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92"}
Jul 27 22:12:18.481144 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:18.481103    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:18.482263 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:18.482242    2578 patch_prober.go:28] interesting pod/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 27 22:12:18.482321 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:18.482287    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 27 22:12:19.481105 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:19.481065    2578 patch_prober.go:28] interesting pod/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 27 22:12:19.481455 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:19.481136    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 27 22:12:20.481539 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:20.481498    2578 patch_prober.go:28] interesting pod/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 27 22:12:20.482023 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:20.481561    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 27 22:12:21.481032 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:21.480991    2578 patch_prober.go:28] interesting pod/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 27 22:12:21.481228 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:21.481048    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 27 22:12:22.485113 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:22.485075    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:22.485535 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:22.485375    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:22.486119 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:22.486098    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:12:31.993634 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:31.993597    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"]
Jul 27 22:12:31.994009 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:31.993973    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy"
Jul 27 22:12:31.994009 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:31.993985    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy"
Jul 27 22:12:31.994009 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:31.993992    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy"
Jul 27 22:12:31.994009 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:31.993998    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy"
Jul 27 22:12:31.994143 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:31.994050    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy"
Jul 27 22:12:31.994143 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:31.994060    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a40085-bcb8-4a62-bddb-2ac6a17517e3" containerName="istio-proxy"
Jul 27 22:12:31.997258 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:31.997239    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.000201 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.000178    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-5e1a2f-serving-cert\""
Jul 27 22:12:32.006546 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.006527    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"]
Jul 27 22:12:32.037264 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.037240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641652df-e127-4073-9bdc-9216902ea36a-maas-api-tls\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.037392 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.037319    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641652df-e127-4073-9bdc-9216902ea36a-openshift-service-ca\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.037392 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.037356    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfxcs\" (UniqueName: \"kubernetes.io/projected/641652df-e127-4073-9bdc-9216902ea36a-kube-api-access-jfxcs\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.138270 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.138243    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641652df-e127-4073-9bdc-9216902ea36a-maas-api-tls\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.138424 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.138295    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641652df-e127-4073-9bdc-9216902ea36a-openshift-service-ca\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.138424 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.138321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfxcs\" (UniqueName: \"kubernetes.io/projected/641652df-e127-4073-9bdc-9216902ea36a-kube-api-access-jfxcs\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.138424 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:12:32.138399    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-serving-cert: secret "maas-api-e2e-api-b-5e1a2f-serving-cert" not found
Jul 27 22:12:32.138535 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:12:32.138471    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/641652df-e127-4073-9bdc-9216902ea36a-maas-api-tls podName:641652df-e127-4073-9bdc-9216902ea36a nodeName:}" failed. No retries permitted until 2026-07-27 22:12:32.638454314 +0000 UTC m=+2323.534263743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/641652df-e127-4073-9bdc-9216902ea36a-maas-api-tls") pod "maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx" (UID: "641652df-e127-4073-9bdc-9216902ea36a") : secret "maas-api-e2e-api-b-5e1a2f-serving-cert" not found
Jul 27 22:12:32.138931 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.138914    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641652df-e127-4073-9bdc-9216902ea36a-openshift-service-ca\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.149798 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.149768    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfxcs\" (UniqueName: \"kubernetes.io/projected/641652df-e127-4073-9bdc-9216902ea36a-kube-api-access-jfxcs\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.641523 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.641487    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641652df-e127-4073-9bdc-9216902ea36a-maas-api-tls\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.643988 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.643964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641652df-e127-4073-9bdc-9216902ea36a-maas-api-tls\") pod \"maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:32.910886 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:32.910855    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:33.047823 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:33.047798    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"]
Jul 27 22:12:33.050003 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:12:33.049977    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641652df_e127_4073_9bdc_9216902ea36a.slice/crio-f492cb898aada7457935544126d8585eaa886c52b167a43e3ff9efe02d50c24e WatchSource:0}: Error finding container f492cb898aada7457935544126d8585eaa886c52b167a43e3ff9efe02d50c24e: Status 404 returned error can't find the container with id f492cb898aada7457935544126d8585eaa886c52b167a43e3ff9efe02d50c24e
Jul 27 22:12:33.694912 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:33.694824    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx" event={"ID":"641652df-e127-4073-9bdc-9216902ea36a","Type":"ContainerStarted","Data":"1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c"}
Jul 27 22:12:33.694912 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:33.694880    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx" event={"ID":"641652df-e127-4073-9bdc-9216902ea36a","Type":"ContainerStarted","Data":"f492cb898aada7457935544126d8585eaa886c52b167a43e3ff9efe02d50c24e"}
Jul 27 22:12:33.695135 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:33.694936    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:12:33.711592 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:33.711543    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx" podStartSLOduration=2.330093996 podStartE2EDuration="2.711529414s" totalImagesPullingTime="381.435418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:12:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:12:33.710094192 +0000 UTC m=+2324.605903679" watchObservedRunningTime="2026-07-27 22:12:33.711529414 +0000 UTC m=+2324.607338865"
Jul 27 22:12:39.706032 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:12:39.706004    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:13:15.387869 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.387832    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"]
Jul 27 22:13:15.388323 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.388044    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d" podUID="d595fe85-07dd-46f2-b258-c8e1e0850caa" containerName="maas-api" containerID="cri-o://5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91" gracePeriod=30
Jul 27 22:13:15.649764 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.649691    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:13:15.781308 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.781280    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d595fe85-07dd-46f2-b258-c8e1e0850caa-openshift-service-ca\") pod \"d595fe85-07dd-46f2-b258-c8e1e0850caa\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") "
Jul 27 22:13:15.781456 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.781358    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d595fe85-07dd-46f2-b258-c8e1e0850caa-maas-api-tls\") pod \"d595fe85-07dd-46f2-b258-c8e1e0850caa\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") "
Jul 27 22:13:15.781456 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.781387    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwfqz\" (UniqueName: \"kubernetes.io/projected/d595fe85-07dd-46f2-b258-c8e1e0850caa-kube-api-access-lwfqz\") pod \"d595fe85-07dd-46f2-b258-c8e1e0850caa\" (UID: \"d595fe85-07dd-46f2-b258-c8e1e0850caa\") "
Jul 27 22:13:15.781753 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.781702    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d595fe85-07dd-46f2-b258-c8e1e0850caa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d595fe85-07dd-46f2-b258-c8e1e0850caa" (UID: "d595fe85-07dd-46f2-b258-c8e1e0850caa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:13:15.783597 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.783568    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d595fe85-07dd-46f2-b258-c8e1e0850caa-kube-api-access-lwfqz" (OuterVolumeSpecName: "kube-api-access-lwfqz") pod "d595fe85-07dd-46f2-b258-c8e1e0850caa" (UID: "d595fe85-07dd-46f2-b258-c8e1e0850caa"). InnerVolumeSpecName "kube-api-access-lwfqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:13:15.783597 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.783584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d595fe85-07dd-46f2-b258-c8e1e0850caa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d595fe85-07dd-46f2-b258-c8e1e0850caa" (UID: "d595fe85-07dd-46f2-b258-c8e1e0850caa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:13:15.881986 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.881963    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d595fe85-07dd-46f2-b258-c8e1e0850caa-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:15.881986 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.881984    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwfqz\" (UniqueName: \"kubernetes.io/projected/d595fe85-07dd-46f2-b258-c8e1e0850caa-kube-api-access-lwfqz\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:15.882116 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.881993    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d595fe85-07dd-46f2-b258-c8e1e0850caa-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:15.896316 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.896291    2578 generic.go:356] "Generic (PLEG): container finished" podID="d595fe85-07dd-46f2-b258-c8e1e0850caa" containerID="5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91" exitCode=0
Jul 27 22:13:15.896394 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.896340    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"
Jul 27 22:13:15.896394 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.896376    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d" event={"ID":"d595fe85-07dd-46f2-b258-c8e1e0850caa","Type":"ContainerDied","Data":"5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91"}
Jul 27 22:13:15.896478 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.896430    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d" event={"ID":"d595fe85-07dd-46f2-b258-c8e1e0850caa","Type":"ContainerDied","Data":"9fd6b38e3908ee9c0215a44962b0feb53876ae72b3fb95b8f4576e0d5e9f74ff"}
Jul 27 22:13:15.896478 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.896452    2578 scope.go:117] "RemoveContainer" containerID="5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91"
Jul 27 22:13:15.905187 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.905172    2578 scope.go:117] "RemoveContainer" containerID="5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91"
Jul 27 22:13:15.905419 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:13:15.905403    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91\": container with ID starting with 5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91 not found: ID does not exist" containerID="5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91"
Jul 27 22:13:15.905464 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.905426    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91"} err="failed to get container status \"5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91\": rpc error: code = NotFound desc = could not find container \"5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91\": container with ID starting with 5009b07c207b1eef2969f23c4c5606956f297aaaa1c5af540235a7005573fd91 not found: ID does not exist"
Jul 27 22:13:15.921755 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.921710    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"]
Jul 27 22:13:15.924457 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:15.924437    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2f19c4-7959965b57-n254d"]
Jul 27 22:13:17.606303 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:17.606269    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d595fe85-07dd-46f2-b258-c8e1e0850caa" path="/var/lib/kubelet/pods/d595fe85-07dd-46f2-b258-c8e1e0850caa/volumes"
Jul 27 22:13:22.167845 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:22.167816    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"]
Jul 27 22:13:22.168395 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:22.168013    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy" containerID="cri-o://672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92" gracePeriod=30
Jul 27 22:13:22.487638 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:22.487551    2578 patch_prober.go:28] interesting pod/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 22:13:22.487638 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:22.487610    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 22:13:27.412500 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.412477    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:13:27.464374 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464308    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvhmp\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-kube-api-access-nvhmp\") pod \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") "
Jul 27 22:13:27.464374 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464340    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-data\") pod \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") "
Jul 27 22:13:27.464374 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464361    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-certs\") pod \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") "
Jul 27 22:13:27.464613 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464396    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-token\") pod \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") "
Jul 27 22:13:27.464613 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464444    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-credential-socket\") pod \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") "
Jul 27 22:13:27.464613 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464468    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-socket\") pod \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") "
Jul 27 22:13:27.464613 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464510    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istiod-ca-cert\") pod \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") "
Jul 27 22:13:27.464613 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464582    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-envoy\") pod \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") "
Jul 27 22:13:27.464902 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464619    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-podinfo\") pod \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\" (UID: \"056dda74-35a2-44fd-9e6e-2d4cb020dd9a\") "
Jul 27 22:13:27.464902 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464648    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "056dda74-35a2-44fd-9e6e-2d4cb020dd9a" (UID: "056dda74-35a2-44fd-9e6e-2d4cb020dd9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:13:27.464902 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464773    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "056dda74-35a2-44fd-9e6e-2d4cb020dd9a" (UID: "056dda74-35a2-44fd-9e6e-2d4cb020dd9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:13:27.465106 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464938    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:27.465106 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.464956    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:27.465106 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.465013    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "056dda74-35a2-44fd-9e6e-2d4cb020dd9a" (UID: "056dda74-35a2-44fd-9e6e-2d4cb020dd9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:13:27.465400 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.465299    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "056dda74-35a2-44fd-9e6e-2d4cb020dd9a" (UID: "056dda74-35a2-44fd-9e6e-2d4cb020dd9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:13:27.465400 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.465308    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-data" (OuterVolumeSpecName: "istio-data") pod "056dda74-35a2-44fd-9e6e-2d4cb020dd9a" (UID: "056dda74-35a2-44fd-9e6e-2d4cb020dd9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:13:27.466860 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.466833    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-kube-api-access-nvhmp" (OuterVolumeSpecName: "kube-api-access-nvhmp") pod "056dda74-35a2-44fd-9e6e-2d4cb020dd9a" (UID: "056dda74-35a2-44fd-9e6e-2d4cb020dd9a"). InnerVolumeSpecName "kube-api-access-nvhmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:13:27.467108 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.467085    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "056dda74-35a2-44fd-9e6e-2d4cb020dd9a" (UID: "056dda74-35a2-44fd-9e6e-2d4cb020dd9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:13:27.467392 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.467369    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "056dda74-35a2-44fd-9e6e-2d4cb020dd9a" (UID: "056dda74-35a2-44fd-9e6e-2d4cb020dd9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:13:27.467472 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.467457    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "056dda74-35a2-44fd-9e6e-2d4cb020dd9a" (UID: "056dda74-35a2-44fd-9e6e-2d4cb020dd9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:13:27.566215 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.566191    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:27.566215 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.566214    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:27.566348 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.566223    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvhmp\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-kube-api-access-nvhmp\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:27.566348 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.566231    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:27.566348 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.566239    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:27.566348 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.566248    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:27.566348 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.566262    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056dda74-35a2-44fd-9e6e-2d4cb020dd9a-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:27.956538 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.956508    2578 generic.go:356] "Generic (PLEG): container finished" podID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerID="672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92" exitCode=0
Jul 27 22:13:27.956684 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.956605    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" event={"ID":"056dda74-35a2-44fd-9e6e-2d4cb020dd9a","Type":"ContainerDied","Data":"672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92"}
Jul 27 22:13:27.956684 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.956634    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"
Jul 27 22:13:27.956804 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.956690    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd" event={"ID":"056dda74-35a2-44fd-9e6e-2d4cb020dd9a","Type":"ContainerDied","Data":"fda48e95f484e13bbce32fcd21941bc9c028864c8d64f5fc63ca8bfe088c4f9a"}
Jul 27 22:13:27.956804 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.956722    2578 scope.go:117] "RemoveContainer" containerID="672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92"
Jul 27 22:13:27.965936 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.965913    2578 scope.go:117] "RemoveContainer" containerID="bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639"
Jul 27 22:13:27.973845 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.973726    2578 scope.go:117] "RemoveContainer" containerID="672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92"
Jul 27 22:13:27.974044 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:13:27.974016    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92\": container with ID starting with 672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92 not found: ID does not exist" containerID="672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92"
Jul 27 22:13:27.974131 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.974054    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92"} err="failed to get container status \"672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92\": rpc error: code = NotFound desc = could not find container \"672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92\": container with ID starting with 672a091ab19aea2408cf2530dceda1413a151e339726cf12b7817071ed309d92 not found: ID does not exist"
Jul 27 22:13:27.974131 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.974079    2578 scope.go:117] "RemoveContainer" containerID="bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639"
Jul 27 22:13:27.974355 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:13:27.974330    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639\": container with ID starting with bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639 not found: ID does not exist" containerID="bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639"
Jul 27 22:13:27.974409 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.974364    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639"} err="failed to get container status \"bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639\": rpc error: code = NotFound desc = could not find container \"bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639\": container with ID starting with bf03769184e483a1c7f39a60b9480be4bf375a4ff4f0bec1e1ac53d791148639 not found: ID does not exist"
Jul 27 22:13:27.980068 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.980035    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"]
Jul 27 22:13:27.983067 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:27.983048    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2f19c4-openshift-default-5d84cd6854-gzlpd"]
Jul 27 22:13:29.605328 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:29.605294    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" path="/var/lib/kubelet/pods/056dda74-35a2-44fd-9e6e-2d4cb020dd9a/volumes"
Jul 27 22:13:31.224134 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.224081    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"]
Jul 27 22:13:31.225616 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.224369    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx" podUID="641652df-e127-4073-9bdc-9216902ea36a" containerName="maas-api" containerID="cri-o://1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c" gracePeriod=30
Jul 27 22:13:31.481940 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.481885    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:13:31.597898 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.597875    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641652df-e127-4073-9bdc-9216902ea36a-openshift-service-ca\") pod \"641652df-e127-4073-9bdc-9216902ea36a\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") "
Jul 27 22:13:31.598025 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.597910    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641652df-e127-4073-9bdc-9216902ea36a-maas-api-tls\") pod \"641652df-e127-4073-9bdc-9216902ea36a\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") "
Jul 27 22:13:31.598025 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.597929    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfxcs\" (UniqueName: \"kubernetes.io/projected/641652df-e127-4073-9bdc-9216902ea36a-kube-api-access-jfxcs\") pod \"641652df-e127-4073-9bdc-9216902ea36a\" (UID: \"641652df-e127-4073-9bdc-9216902ea36a\") "
Jul 27 22:13:31.598260 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.598240    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/641652df-e127-4073-9bdc-9216902ea36a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "641652df-e127-4073-9bdc-9216902ea36a" (UID: "641652df-e127-4073-9bdc-9216902ea36a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:13:31.600304 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.600272    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/641652df-e127-4073-9bdc-9216902ea36a-kube-api-access-jfxcs" (OuterVolumeSpecName: "kube-api-access-jfxcs") pod "641652df-e127-4073-9bdc-9216902ea36a" (UID: "641652df-e127-4073-9bdc-9216902ea36a"). InnerVolumeSpecName "kube-api-access-jfxcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:13:31.600409 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.600358    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/641652df-e127-4073-9bdc-9216902ea36a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "641652df-e127-4073-9bdc-9216902ea36a" (UID: "641652df-e127-4073-9bdc-9216902ea36a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:13:31.698981 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.698948    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/641652df-e127-4073-9bdc-9216902ea36a-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:31.698981 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.698971    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/641652df-e127-4073-9bdc-9216902ea36a-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:31.698981 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.698981    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfxcs\" (UniqueName: \"kubernetes.io/projected/641652df-e127-4073-9bdc-9216902ea36a-kube-api-access-jfxcs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:13:31.979643 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.979604    2578 generic.go:356] "Generic (PLEG): container finished" podID="641652df-e127-4073-9bdc-9216902ea36a" containerID="1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c" exitCode=0
Jul 27 22:13:31.979834 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.979658    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"
Jul 27 22:13:31.979834 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.979696    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx" event={"ID":"641652df-e127-4073-9bdc-9216902ea36a","Type":"ContainerDied","Data":"1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c"}
Jul 27 22:13:31.979834 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.979775    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx" event={"ID":"641652df-e127-4073-9bdc-9216902ea36a","Type":"ContainerDied","Data":"f492cb898aada7457935544126d8585eaa886c52b167a43e3ff9efe02d50c24e"}
Jul 27 22:13:31.979834 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.979804    2578 scope.go:117] "RemoveContainer" containerID="1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c"
Jul 27 22:13:31.988697 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.988680    2578 scope.go:117] "RemoveContainer" containerID="1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c"
Jul 27 22:13:31.989043 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:13:31.989020    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c\": container with ID starting with 1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c not found: ID does not exist" containerID="1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c"
Jul 27 22:13:31.989128 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:31.989057    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c"} err="failed to get container status \"1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c\": rpc error: code = NotFound desc = could not find container \"1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c\": container with ID starting with 1f4bdcab4fda55f80922be909d2d08e674965e8a19ba323c38481f37730a9b0c not found: ID does not exist"
Jul 27 22:13:32.001174 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:32.001153    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"]
Jul 27 22:13:32.006521 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:32.006500    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e1a2f-b7859567f-8vjlx"]
Jul 27 22:13:33.605322 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:33.605286    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="641652df-e127-4073-9bdc-9216902ea36a" path="/var/lib/kubelet/pods/641652df-e127-4073-9bdc-9216902ea36a/volumes"
Jul 27 22:13:39.322283 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322251    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"]
Jul 27 22:13:39.322654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322570    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d595fe85-07dd-46f2-b258-c8e1e0850caa" containerName="maas-api"
Jul 27 22:13:39.322654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322580    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d595fe85-07dd-46f2-b258-c8e1e0850caa" containerName="maas-api"
Jul 27 22:13:39.322654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322591    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641652df-e127-4073-9bdc-9216902ea36a" containerName="maas-api"
Jul 27 22:13:39.322654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322596    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="641652df-e127-4073-9bdc-9216902ea36a" containerName="maas-api"
Jul 27 22:13:39.322654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322603    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy"
Jul 27 22:13:39.322654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322609    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy"
Jul 27 22:13:39.322976 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322674    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy"
Jul 27 22:13:39.322976 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322682    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="641652df-e127-4073-9bdc-9216902ea36a" containerName="maas-api"
Jul 27 22:13:39.322976 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.322689    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d595fe85-07dd-46f2-b258-c8e1e0850caa" containerName="maas-api"
Jul 27 22:13:39.327002 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.326986    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.330008 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.329984    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-94450e-openshift-default-dockercfg-fnqgl\""
Jul 27 22:13:39.354354 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.354320    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"]
Jul 27 22:13:39.454796 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.454768    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-certs\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.454914 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.454799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-envoy\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.454914 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.454815    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/978e5de1-fc85-4157-a53c-213d765a770e-istio-podinfo\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.454914 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.454836    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/978e5de1-fc85-4157-a53c-213d765a770e-istiod-ca-cert\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.454914 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.454853    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-socket\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.454914 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.454879    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-istio-token\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.455076 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.454920    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzwzr\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-kube-api-access-dzwzr\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.455076 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.454949    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-data\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.455076 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.454971    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-credential-socket\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556234 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzwzr\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-kube-api-access-dzwzr\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556339 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-data\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556339 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-credential-socket\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556339 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-certs\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556339 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-envoy\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556541 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556360    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/978e5de1-fc85-4157-a53c-213d765a770e-istio-podinfo\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556541 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/978e5de1-fc85-4157-a53c-213d765a770e-istiod-ca-cert\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556541 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-socket\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556541 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-istio-token\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556770 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556677    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-data\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556770 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556726    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-certs\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556885 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556817    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-socket\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.556966 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.556947    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-credential-socket\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.557094 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.557074    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/978e5de1-fc85-4157-a53c-213d765a770e-istiod-ca-cert\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.558917 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.558894    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-envoy\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.559109 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.559090    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/978e5de1-fc85-4157-a53c-213d765a770e-istio-podinfo\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.565991 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.565958    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzwzr\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-kube-api-access-dzwzr\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.566088 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.566064    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-istio-token\") pod \"e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") " pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.637567 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.637505    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:39.781641 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.781614    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"]
Jul 27 22:13:39.784518 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:13:39.784427    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978e5de1_fc85_4157_a53c_213d765a770e.slice/crio-436e8866f1c7a944c2db6849ee955e890c2d0c9e27de5e944b84106daa7f86e7 WatchSource:0}: Error finding container 436e8866f1c7a944c2db6849ee955e890c2d0c9e27de5e944b84106daa7f86e7: Status 404 returned error can't find the container with id 436e8866f1c7a944c2db6849ee955e890c2d0c9e27de5e944b84106daa7f86e7
Jul 27 22:13:39.787020 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.786981    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:13:39.787129 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.787052    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:13:39.787129 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:39.787088    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:13:40.022476 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:40.022439    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" event={"ID":"978e5de1-fc85-4157-a53c-213d765a770e","Type":"ContainerStarted","Data":"9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d"}
Jul 27 22:13:40.022656 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:40.022508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" event={"ID":"978e5de1-fc85-4157-a53c-213d765a770e","Type":"ContainerStarted","Data":"436e8866f1c7a944c2db6849ee955e890c2d0c9e27de5e944b84106daa7f86e7"}
Jul 27 22:13:40.047870 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:40.047812    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" podStartSLOduration=1.047792917 podStartE2EDuration="1.047792917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:13:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:13:40.04665041 +0000 UTC m=+2390.942459874" watchObservedRunningTime="2026-07-27 22:13:40.047792917 +0000 UTC m=+2390.943602369"
Jul 27 22:13:40.637900 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:40.637869    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:40.642632 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:40.642603    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:41.027797 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:41.027769    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:41.028669 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:41.028651    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:42.222428 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.222397    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"]
Jul 27 22:13:42.222812 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.222791    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy"
Jul 27 22:13:42.222812 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.222808    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy"
Jul 27 22:13:42.222950 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.222885    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="056dda74-35a2-44fd-9e6e-2d4cb020dd9a" containerName="istio-proxy"
Jul 27 22:13:42.225925 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.225911    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.228979 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.228955    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-94450e-serving-cert\""
Jul 27 22:13:42.247510 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.247486    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"]
Jul 27 22:13:42.278642 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.278619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af12b272-3cff-4da8-b157-88f822e97792-maas-api-tls\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.278784 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.278661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7lbc\" (UniqueName: \"kubernetes.io/projected/af12b272-3cff-4da8-b157-88f822e97792-kube-api-access-f7lbc\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.278784 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.278694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af12b272-3cff-4da8-b157-88f822e97792-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.379646 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.379606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af12b272-3cff-4da8-b157-88f822e97792-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.379831 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.379691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af12b272-3cff-4da8-b157-88f822e97792-maas-api-tls\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.379831 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.379744    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7lbc\" (UniqueName: \"kubernetes.io/projected/af12b272-3cff-4da8-b157-88f822e97792-kube-api-access-f7lbc\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.379947 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:13:42.379850    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-serving-cert: secret "maas-api-e2e-shared-a-94450e-serving-cert" not found
Jul 27 22:13:42.379947 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:13:42.379925    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af12b272-3cff-4da8-b157-88f822e97792-maas-api-tls podName:af12b272-3cff-4da8-b157-88f822e97792 nodeName:}" failed. No retries permitted until 2026-07-27 22:13:42.879901423 +0000 UTC m=+2393.775710854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/af12b272-3cff-4da8-b157-88f822e97792-maas-api-tls") pod "maas-api-e2e-shared-a-94450e-6996b76556-qvrmc" (UID: "af12b272-3cff-4da8-b157-88f822e97792") : secret "maas-api-e2e-shared-a-94450e-serving-cert" not found
Jul 27 22:13:42.380321 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.380294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af12b272-3cff-4da8-b157-88f822e97792-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.390383 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.390364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7lbc\" (UniqueName: \"kubernetes.io/projected/af12b272-3cff-4da8-b157-88f822e97792-kube-api-access-f7lbc\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.883542 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.883513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af12b272-3cff-4da8-b157-88f822e97792-maas-api-tls\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:42.886172 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:42.886147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af12b272-3cff-4da8-b157-88f822e97792-maas-api-tls\") pod \"maas-api-e2e-shared-a-94450e-6996b76556-qvrmc\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:43.136437 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:43.136360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:43.305590 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:43.305561    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"]
Jul 27 22:13:43.307543 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:13:43.307519    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf12b272_3cff_4da8_b157_88f822e97792.slice/crio-24286a085b82d44d6a624fe99ce62f886ca02967f4acd9539bf5b8fba73354fa WatchSource:0}: Error finding container 24286a085b82d44d6a624fe99ce62f886ca02967f4acd9539bf5b8fba73354fa: Status 404 returned error can't find the container with id 24286a085b82d44d6a624fe99ce62f886ca02967f4acd9539bf5b8fba73354fa
Jul 27 22:13:44.044525 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:44.044438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc" event={"ID":"af12b272-3cff-4da8-b157-88f822e97792","Type":"ContainerStarted","Data":"25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32"}
Jul 27 22:13:44.044652 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:44.044619    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:44.044696 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:44.044649    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc" event={"ID":"af12b272-3cff-4da8-b157-88f822e97792","Type":"ContainerStarted","Data":"24286a085b82d44d6a624fe99ce62f886ca02967f4acd9539bf5b8fba73354fa"}
Jul 27 22:13:44.075587 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:44.075538    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc" podStartSLOduration=1.627160227 podStartE2EDuration="2.07552552s" totalImagesPullingTime="448.365293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:13:44.074542654 +0000 UTC m=+2394.970352106" watchObservedRunningTime="2026-07-27 22:13:44.07552552 +0000 UTC m=+2394.971334971"
Jul 27 22:13:49.648665 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:49.648637    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:13:49.685877 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:49.685856    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:13:50.060605 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:50.060572    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:13:55.104185 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:55.104090    2578 generic.go:356] "Generic (PLEG): container finished" podID="978e5de1-fc85-4157-a53c-213d765a770e" containerID="9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d" exitCode=0
Jul 27 22:13:55.104185 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:55.104168    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" event={"ID":"978e5de1-fc85-4157-a53c-213d765a770e","Type":"ContainerDied","Data":"9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d"}
Jul 27 22:13:55.104574 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:55.104549    2578 scope.go:117] "RemoveContainer" containerID="9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d"
Jul 27 22:13:55.105493 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:55.105469    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:13:55.105554 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:55.105540    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:13:55.105591 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:55.105575    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:13:56.028647 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:56.028610    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:56.110698 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:56.110661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" event={"ID":"978e5de1-fc85-4157-a53c-213d765a770e","Type":"ContainerStarted","Data":"4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda"}
Jul 27 22:13:56.638192 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:56.638162    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:13:56.639031 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:56.639001    2578 patch_prober.go:28] interesting pod/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 27 22:13:56.639154 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:56.639053    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" podUID="978e5de1-fc85-4157-a53c-213d765a770e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 27 22:13:57.638334 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:57.638300    2578 patch_prober.go:28] interesting pod/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 27 22:13:57.638776 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:57.638354    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" podUID="978e5de1-fc85-4157-a53c-213d765a770e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 27 22:13:58.638867 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:58.638835    2578 patch_prober.go:28] interesting pod/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 27 22:13:58.639239 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:58.638887    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" podUID="978e5de1-fc85-4157-a53c-213d765a770e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 27 22:13:59.638005 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:59.637970    2578 patch_prober.go:28] interesting pod/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 27 22:13:59.638213 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:13:59.638026    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" podUID="978e5de1-fc85-4157-a53c-213d765a770e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 27 22:14:00.642307 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:00.642276    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:14:00.642699 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:00.642681    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:14:00.643546 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:00.643523    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:14:07.874278 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.874246    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"]
Jul 27 22:14:07.877003 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.876983    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:07.880277 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.880253    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-0cb892-openshift-default-dockercfg-ll967\""
Jul 27 22:14:07.929054 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.929022    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"]
Jul 27 22:14:07.994112 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.994077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-podinfo\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:07.994285 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.994122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-certs\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:07.994285 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.994141    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-socket\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:07.994285 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.994241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-credential-socket\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:07.994285 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.994275    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istiod-ca-cert\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:07.994480 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.994352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-token\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:07.994480 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.994374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-data\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:07.994480 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.994393    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88hjz\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-kube-api-access-88hjz\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:07.994480 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:07.994422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-envoy\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.095434 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095404    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-token\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.095555 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095438    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-data\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.095555 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095458    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88hjz\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-kube-api-access-88hjz\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.095555 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-envoy\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.095555 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-podinfo\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.095806 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-certs\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.095806 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-socket\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.095806 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-credential-socket\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.095806 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095648    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istiod-ca-cert\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.096028 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-data\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.096028 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.095996    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-socket\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.096149 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.096055    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-certs\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.096149 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.096101    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-credential-socket\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.096346 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.096328    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istiod-ca-cert\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.098041 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.098014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-envoy\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.098141 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.098117    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-podinfo\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.114905 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.114870    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-token\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.116440 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.116418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88hjz\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-kube-api-access-88hjz\") pod \"e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") " pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.186461 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.186429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:08.341560 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.341536    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"]
Jul 27 22:14:08.342441 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:14:08.342419    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode835e6a0_575a_42cd_93f3_aa5f86aa67c7.slice/crio-7fb30ef7d07a512a549092f98e2bb7dcbf30311fa287e475442c43027b026587 WatchSource:0}: Error finding container 7fb30ef7d07a512a549092f98e2bb7dcbf30311fa287e475442c43027b026587: Status 404 returned error can't find the container with id 7fb30ef7d07a512a549092f98e2bb7dcbf30311fa287e475442c43027b026587
Jul 27 22:14:08.344498 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.344465    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:14:08.344576 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.344530    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:14:08.344576 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:08.344555    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:14:09.174192 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:09.174153    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" event={"ID":"e835e6a0-575a-42cd-93f3-aa5f86aa67c7","Type":"ContainerStarted","Data":"dd5e7433c8d734b63ad56ef2ff7a98b081ce8246166e0bced69f682e6cc489c7"}
Jul 27 22:14:09.174658 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:09.174213    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" event={"ID":"e835e6a0-575a-42cd-93f3-aa5f86aa67c7","Type":"ContainerStarted","Data":"7fb30ef7d07a512a549092f98e2bb7dcbf30311fa287e475442c43027b026587"}
Jul 27 22:14:09.186938 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:09.186912    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:09.191664 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:09.191645    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:09.211294 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:09.211255    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" podStartSLOduration=2.211240865 podStartE2EDuration="2.211240865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:14:09.210316512 +0000 UTC m=+2420.106125999" watchObservedRunningTime="2026-07-27 22:14:09.211240865 +0000 UTC m=+2420.107050316"
Jul 27 22:14:10.178931 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:10.178856    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:10.179626 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:10.179608    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:12.389086 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.389052    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"]
Jul 27 22:14:12.393487 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.393447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.397385 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.397362    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-0cb892-serving-cert\""
Jul 27 22:14:12.422952 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.422926    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"]
Jul 27 22:14:12.531287 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.531258    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5dks\" (UniqueName: \"kubernetes.io/projected/aba0ed7a-896c-48d2-a79c-71da71d534ae-kube-api-access-m5dks\") pod \"maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.531454 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.531340    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aba0ed7a-896c-48d2-a79c-71da71d534ae-maas-api-tls\") pod \"maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.531454 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.531363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aba0ed7a-896c-48d2-a79c-71da71d534ae-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.632504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.632477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aba0ed7a-896c-48d2-a79c-71da71d534ae-maas-api-tls\") pod \"maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.632651 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.632513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aba0ed7a-896c-48d2-a79c-71da71d534ae-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.632651 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.632550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5dks\" (UniqueName: \"kubernetes.io/projected/aba0ed7a-896c-48d2-a79c-71da71d534ae-kube-api-access-m5dks\") pod \"maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.633240 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.633155    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aba0ed7a-896c-48d2-a79c-71da71d534ae-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.634889 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.634870    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aba0ed7a-896c-48d2-a79c-71da71d534ae-maas-api-tls\") pod \"maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.644718 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.644668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5dks\" (UniqueName: \"kubernetes.io/projected/aba0ed7a-896c-48d2-a79c-71da71d534ae-kube-api-access-m5dks\") pod \"maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.704657 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.704621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:12.858848 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:12.858819    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"]
Jul 27 22:14:12.865376 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:14:12.865351    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba0ed7a_896c_48d2_a79c_71da71d534ae.slice/crio-ea264b911a28cdb90799e9cf58c2d94328ef060f6ba4db1f99b30b53d35a4254 WatchSource:0}: Error finding container ea264b911a28cdb90799e9cf58c2d94328ef060f6ba4db1f99b30b53d35a4254: Status 404 returned error can't find the container with id ea264b911a28cdb90799e9cf58c2d94328ef060f6ba4db1f99b30b53d35a4254
Jul 27 22:14:13.194065 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:13.194035    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7" event={"ID":"aba0ed7a-896c-48d2-a79c-71da71d534ae","Type":"ContainerStarted","Data":"ea264b911a28cdb90799e9cf58c2d94328ef060f6ba4db1f99b30b53d35a4254"}
Jul 27 22:14:14.200771 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:14.200714    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7" event={"ID":"aba0ed7a-896c-48d2-a79c-71da71d534ae","Type":"ContainerStarted","Data":"3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c"}
Jul 27 22:14:14.201143 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:14.200840    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:14.234773 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:14.234702    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7" podStartSLOduration=1.8601307889999998 podStartE2EDuration="2.234689305s" totalImagesPullingTime="374.558516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:14:14.232945143 +0000 UTC m=+2425.128754594" watchObservedRunningTime="2026-07-27 22:14:14.234689305 +0000 UTC m=+2425.130498755"
Jul 27 22:14:20.212259 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:20.212229    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:14:26.220891 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:26.220845    2578 patch_prober.go:28] interesting pod/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 27 22:14:26.221373 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:26.220918    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" podUID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 27 22:14:27.265342 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:27.265310    2578 generic.go:356] "Generic (PLEG): container finished" podID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerID="dd5e7433c8d734b63ad56ef2ff7a98b081ce8246166e0bced69f682e6cc489c7" exitCode=0
Jul 27 22:14:27.265778 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:27.265359    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" event={"ID":"e835e6a0-575a-42cd-93f3-aa5f86aa67c7","Type":"ContainerDied","Data":"dd5e7433c8d734b63ad56ef2ff7a98b081ce8246166e0bced69f682e6cc489c7"}
Jul 27 22:14:27.265778 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:27.265702    2578 scope.go:117] "RemoveContainer" containerID="dd5e7433c8d734b63ad56ef2ff7a98b081ce8246166e0bced69f682e6cc489c7"
Jul 27 22:14:27.266687 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:27.266659    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:14:27.266771 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:27.266726    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:14:27.266820 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:27.266793    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:14:28.277388 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:28.277335    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" event={"ID":"e835e6a0-575a-42cd-93f3-aa5f86aa67c7","Type":"ContainerStarted","Data":"760ce2c5b8026a7fbaafbf92df40d44a783543110b10c7ecca47e8ffc8c72366"}
Jul 27 22:14:29.187052 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:29.187018    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:29.187232 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:29.187203    2578 patch_prober.go:28] interesting pod/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 27 22:14:29.187280 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:29.187252    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" podUID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 27 22:14:30.187448 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:30.187410    2578 patch_prober.go:28] interesting pod/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 27 22:14:30.187967 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:30.187483    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" podUID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 27 22:14:31.187580 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:31.187542    2578 patch_prober.go:28] interesting pod/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 27 22:14:31.187979 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:31.187603    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" podUID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 27 22:14:32.190955 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:32.190928    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:32.191415 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:32.191205    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:14:32.191913 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:14:32.191899    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:18:49.701114 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:18:49.701045    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:18:49.715725 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:18:49.715701    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:23:49.731332 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:49.731301    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:23:49.747573 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:49.747545    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:23:50.227317 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.227279    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"]
Jul 27 22:23:50.230928 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.230906    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.233760 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.233743    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-dd2178-serving-cert\""
Jul 27 22:23:50.241772 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.241111    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"]
Jul 27 22:23:50.342202 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.342169    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld94z\" (UniqueName: \"kubernetes.io/projected/e1182234-1160-4b2f-80e3-07c3f6cc3af6-kube-api-access-ld94z\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.342354 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.342209    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1182234-1160-4b2f-80e3-07c3f6cc3af6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.342354 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.342231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1182234-1160-4b2f-80e3-07c3f6cc3af6-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.443661 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.443620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1182234-1160-4b2f-80e3-07c3f6cc3af6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.443661 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.443662    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1182234-1160-4b2f-80e3-07c3f6cc3af6-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.443944 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:23:50.443807    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-serving-cert: secret "maas-api-e2e-ipp-a-dd2178-serving-cert" not found
Jul 27 22:23:50.443944 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.443836    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld94z\" (UniqueName: \"kubernetes.io/projected/e1182234-1160-4b2f-80e3-07c3f6cc3af6-kube-api-access-ld94z\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.443944 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:23:50.443909    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1182234-1160-4b2f-80e3-07c3f6cc3af6-maas-api-tls podName:e1182234-1160-4b2f-80e3-07c3f6cc3af6 nodeName:}" failed. No retries permitted until 2026-07-27 22:23:50.943886473 +0000 UTC m=+3001.839695901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1182234-1160-4b2f-80e3-07c3f6cc3af6-maas-api-tls") pod "maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb" (UID: "e1182234-1160-4b2f-80e3-07c3f6cc3af6") : secret "maas-api-e2e-ipp-a-dd2178-serving-cert" not found
Jul 27 22:23:50.444395 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.444379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1182234-1160-4b2f-80e3-07c3f6cc3af6-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.457252 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.457221    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld94z\" (UniqueName: \"kubernetes.io/projected/e1182234-1160-4b2f-80e3-07c3f6cc3af6-kube-api-access-ld94z\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.948068 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.948024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1182234-1160-4b2f-80e3-07c3f6cc3af6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:50.950756 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:50.950714    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1182234-1160-4b2f-80e3-07c3f6cc3af6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:51.142711 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:51.142681    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:51.284621 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:51.284598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"]
Jul 27 22:23:51.286998 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:23:51.286972    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1182234_1160_4b2f_80e3_07c3f6cc3af6.slice/crio-eaa64f54da83d5f5bf1fd4de37022a926573961dc0ff9f5b0c551282d2ed5a3d WatchSource:0}: Error finding container eaa64f54da83d5f5bf1fd4de37022a926573961dc0ff9f5b0c551282d2ed5a3d: Status 404 returned error can't find the container with id eaa64f54da83d5f5bf1fd4de37022a926573961dc0ff9f5b0c551282d2ed5a3d
Jul 27 22:23:51.288666 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:51.288649    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 22:23:52.056807 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:52.056770    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb" event={"ID":"e1182234-1160-4b2f-80e3-07c3f6cc3af6","Type":"ContainerStarted","Data":"11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8"}
Jul 27 22:23:52.057285 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:52.056835    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb" event={"ID":"e1182234-1160-4b2f-80e3-07c3f6cc3af6","Type":"ContainerStarted","Data":"eaa64f54da83d5f5bf1fd4de37022a926573961dc0ff9f5b0c551282d2ed5a3d"}
Jul 27 22:23:52.057285 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:52.056862    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:23:52.077395 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:52.077347    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb" podStartSLOduration=1.5027948960000002 podStartE2EDuration="2.077329804s" totalImagesPullingTime="574.534908ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:23:52.074610971 +0000 UTC m=+3002.970420428" watchObservedRunningTime="2026-07-27 22:23:52.077329804 +0000 UTC m=+3002.973139256"
Jul 27 22:23:58.068560 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:23:58.068528    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:24:10.627417 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.627384    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"]
Jul 27 22:24:10.630968 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.630947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:10.634200 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.634178    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-54963f-serving-cert\""
Jul 27 22:24:10.641848 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.641827    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"]
Jul 27 22:24:10.731546 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.731503    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-727bx\" (UniqueName: \"kubernetes.io/projected/a6d45b6f-f05c-48b2-bc62-9e2612945f97-kube-api-access-727bx\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:10.731764 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.731593    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6d45b6f-f05c-48b2-bc62-9e2612945f97-maas-api-tls\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:10.731764 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.731645    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d45b6f-f05c-48b2-bc62-9e2612945f97-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:10.832397 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.832364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6d45b6f-f05c-48b2-bc62-9e2612945f97-maas-api-tls\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:10.832565 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.832408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d45b6f-f05c-48b2-bc62-9e2612945f97-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:10.832565 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.832471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-727bx\" (UniqueName: \"kubernetes.io/projected/a6d45b6f-f05c-48b2-bc62-9e2612945f97-kube-api-access-727bx\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:10.832565 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:24:10.832510    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-serving-cert: secret "maas-api-e2e-ipp-b-54963f-serving-cert" not found
Jul 27 22:24:10.832688 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:24:10.832569    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d45b6f-f05c-48b2-bc62-9e2612945f97-maas-api-tls podName:a6d45b6f-f05c-48b2-bc62-9e2612945f97 nodeName:}" failed. No retries permitted until 2026-07-27 22:24:11.332551814 +0000 UTC m=+3022.228361247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a6d45b6f-f05c-48b2-bc62-9e2612945f97-maas-api-tls") pod "maas-api-e2e-ipp-b-54963f-659df8457-spn4k" (UID: "a6d45b6f-f05c-48b2-bc62-9e2612945f97") : secret "maas-api-e2e-ipp-b-54963f-serving-cert" not found
Jul 27 22:24:10.833106 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.833087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d45b6f-f05c-48b2-bc62-9e2612945f97-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:10.842079 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:10.842056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-727bx\" (UniqueName: \"kubernetes.io/projected/a6d45b6f-f05c-48b2-bc62-9e2612945f97-kube-api-access-727bx\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:11.338357 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:11.338319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6d45b6f-f05c-48b2-bc62-9e2612945f97-maas-api-tls\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:11.341165 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:11.341140    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6d45b6f-f05c-48b2-bc62-9e2612945f97-maas-api-tls\") pod \"maas-api-e2e-ipp-b-54963f-659df8457-spn4k\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:11.543724 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:11.543681    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:11.695194 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:11.695169    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"]
Jul 27 22:24:11.696707 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:24:11.696678    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d45b6f_f05c_48b2_bc62_9e2612945f97.slice/crio-df659e2723b33717ae654bc0842ca9e0a6e7ee1dced97e661b9b1fa75bfb709b WatchSource:0}: Error finding container df659e2723b33717ae654bc0842ca9e0a6e7ee1dced97e661b9b1fa75bfb709b: Status 404 returned error can't find the container with id df659e2723b33717ae654bc0842ca9e0a6e7ee1dced97e661b9b1fa75bfb709b
Jul 27 22:24:12.158349 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:12.158318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k" event={"ID":"a6d45b6f-f05c-48b2-bc62-9e2612945f97","Type":"ContainerStarted","Data":"36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf"}
Jul 27 22:24:12.158464 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:12.158381    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k" event={"ID":"a6d45b6f-f05c-48b2-bc62-9e2612945f97","Type":"ContainerStarted","Data":"df659e2723b33717ae654bc0842ca9e0a6e7ee1dced97e661b9b1fa75bfb709b"}
Jul 27 22:24:12.158464 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:12.158415    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:12.179119 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:12.179071    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k" podStartSLOduration=1.792885547 podStartE2EDuration="2.179055175s" totalImagesPullingTime="386.169628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:24:12.17629269 +0000 UTC m=+3023.072102142" watchObservedRunningTime="2026-07-27 22:24:12.179055175 +0000 UTC m=+3023.074864626"
Jul 27 22:24:18.175431 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:18.175397    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:24:30.415784 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:24:30.415757    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-885jp_05798666-dd9a-440c-98c8-280baf18deb3/payload-processing/0.log"
Jul 27 22:25:53.663463 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:53.663436    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-885jp_05798666-dd9a-440c-98c8-280baf18deb3/payload-processing/0.log"
Jul 27 22:25:54.470748 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.470694    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"]
Jul 27 22:25:54.474633 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.474611    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.479849 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.479823    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-eab778-openshift-default-dockercfg-k8dsx\""
Jul 27 22:25:54.562995 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.562957    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-certs\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.563160 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.563008    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2da9694-a921-472e-8392-74b043696b9d-istio-podinfo\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.563160 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.563034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-istio-token\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.563160 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.563064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-socket\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.563160 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.563151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-credential-socket\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.563368 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.563175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2da9694-a921-472e-8392-74b043696b9d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.563368 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.563197    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blznk\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-kube-api-access-blznk\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.563368 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.563234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-data\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.563368 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.563289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-envoy\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.603603 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.603486    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"]
Jul 27 22:25:54.664037 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.663999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-certs\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2da9694-a921-472e-8392-74b043696b9d-istio-podinfo\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664088    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-istio-token\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-socket\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-credential-socket\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2da9694-a921-472e-8392-74b043696b9d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blznk\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-kube-api-access-blznk\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664420    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-certs\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-data\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-envoy\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.664504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-socket\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.665080 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-credential-socket\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.665080 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.664678    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-data\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.665080 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.665007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2da9694-a921-472e-8392-74b043696b9d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.666631 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.666603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-envoy\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.666787 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.666703    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2da9694-a921-472e-8392-74b043696b9d-istio-podinfo\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.708248 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.708227    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-istio-token\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.722120 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.722056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blznk\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-kube-api-access-blznk\") pod \"e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") " pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.785408 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.785376    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:54.939169 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.939138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"]
Jul 27 22:25:54.942084 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:25:54.942059    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2da9694_a921_472e_8392_74b043696b9d.slice/crio-a21207d11056e981a9cc1b907a273ebad0b5b4569cc8836967f7168abfdfa67f WatchSource:0}: Error finding container a21207d11056e981a9cc1b907a273ebad0b5b4569cc8836967f7168abfdfa67f: Status 404 returned error can't find the container with id a21207d11056e981a9cc1b907a273ebad0b5b4569cc8836967f7168abfdfa67f
Jul 27 22:25:54.944211 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.944183    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:25:54.944270 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.944243    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:25:54.944306 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:54.944271    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:25:55.685111 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:55.685068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" event={"ID":"d2da9694-a921-472e-8392-74b043696b9d","Type":"ContainerStarted","Data":"67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251"}
Jul 27 22:25:55.685535 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:55.685130    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" event={"ID":"d2da9694-a921-472e-8392-74b043696b9d","Type":"ContainerStarted","Data":"a21207d11056e981a9cc1b907a273ebad0b5b4569cc8836967f7168abfdfa67f"}
Jul 27 22:25:55.785480 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:55.785455    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:55.790408 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:55.790387    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:55.821140 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:55.821095    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" podStartSLOduration=1.821081522 podStartE2EDuration="1.821081522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:25:55.724846747 +0000 UTC m=+3126.620656198" watchObservedRunningTime="2026-07-27 22:25:55.821081522 +0000 UTC m=+3126.716890974"
Jul 27 22:25:56.690175 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:56.690137    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:25:56.691011 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:25:56.690988    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:26:01.028327 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.028289    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"]
Jul 27 22:26:01.031860 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.031842    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.034959 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.034939    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-eab778-serving-cert\""
Jul 27 22:26:01.043971 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.043949    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"]
Jul 27 22:26:01.128331 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.128295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76hrw\" (UniqueName: \"kubernetes.io/projected/70a873c7-e272-486c-88c2-460eb727ea40-kube-api-access-76hrw\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.128499 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.128375    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a873c7-e272-486c-88c2-460eb727ea40-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.128499 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.128403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.229716 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.229679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76hrw\" (UniqueName: \"kubernetes.io/projected/70a873c7-e272-486c-88c2-460eb727ea40-kube-api-access-76hrw\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.229945 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.229795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a873c7-e272-486c-88c2-460eb727ea40-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.229945 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.229820    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.229945 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:26:01.229931    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-serving-cert: secret "maas-api-e2e-ipp-cleanup-eab778-serving-cert" not found
Jul 27 22:26:01.230110 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:26:01.229990    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls podName:70a873c7-e272-486c-88c2-460eb727ea40 nodeName:}" failed. No retries permitted until 2026-07-27 22:26:01.729974998 +0000 UTC m=+3132.625784427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh" (UID: "70a873c7-e272-486c-88c2-460eb727ea40") : secret "maas-api-e2e-ipp-cleanup-eab778-serving-cert" not found
Jul 27 22:26:01.230436 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.230415    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a873c7-e272-486c-88c2-460eb727ea40-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.238877 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.238857    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76hrw\" (UniqueName: \"kubernetes.io/projected/70a873c7-e272-486c-88c2-460eb727ea40-kube-api-access-76hrw\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.734062 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:01.734028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:01.734230 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:26:01.734213    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-serving-cert: secret "maas-api-e2e-ipp-cleanup-eab778-serving-cert" not found
Jul 27 22:26:01.734328 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:26:01.734303    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls podName:70a873c7-e272-486c-88c2-460eb727ea40 nodeName:}" failed. No retries permitted until 2026-07-27 22:26:02.734280324 +0000 UTC m=+3133.630089754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh" (UID: "70a873c7-e272-486c-88c2-460eb727ea40") : secret "maas-api-e2e-ipp-cleanup-eab778-serving-cert" not found
Jul 27 22:26:02.743256 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:02.743219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:02.745901 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:02.745869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:02.843651 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:02.843615    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:02.982501 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:02.982477    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"]
Jul 27 22:26:02.984506 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:26:02.984481    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a873c7_e272_486c_88c2_460eb727ea40.slice/crio-905c6bbbe06c50be22631686f132d73ccfdc406cf3392380f2a24c9e5bb44587 WatchSource:0}: Error finding container 905c6bbbe06c50be22631686f132d73ccfdc406cf3392380f2a24c9e5bb44587: Status 404 returned error can't find the container with id 905c6bbbe06c50be22631686f132d73ccfdc406cf3392380f2a24c9e5bb44587
Jul 27 22:26:03.725503 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:03.725464    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh" event={"ID":"70a873c7-e272-486c-88c2-460eb727ea40","Type":"ContainerStarted","Data":"97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933"}
Jul 27 22:26:03.725677 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:03.725515    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh" event={"ID":"70a873c7-e272-486c-88c2-460eb727ea40","Type":"ContainerStarted","Data":"905c6bbbe06c50be22631686f132d73ccfdc406cf3392380f2a24c9e5bb44587"}
Jul 27 22:26:03.725677 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:03.725599    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:03.763420 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:03.763372    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh" podStartSLOduration=2.228715223 podStartE2EDuration="2.763358001s" totalImagesPullingTime="534.642778ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:26:03.761972977 +0000 UTC m=+3134.657782440" watchObservedRunningTime="2026-07-27 22:26:03.763358001 +0000 UTC m=+3134.659167453"
Jul 27 22:26:09.736155 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:09.736112    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:15.794750 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:15.794698    2578 generic.go:356] "Generic (PLEG): container finished" podID="d2da9694-a921-472e-8392-74b043696b9d" containerID="67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251" exitCode=0
Jul 27 22:26:15.795166 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:15.794772    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" event={"ID":"d2da9694-a921-472e-8392-74b043696b9d","Type":"ContainerDied","Data":"67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251"}
Jul 27 22:26:15.795166 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:15.795126    2578 scope.go:117] "RemoveContainer" containerID="67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251"
Jul 27 22:26:15.796370 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:15.796345    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:26:15.796436 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:15.796413    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:26:15.796473 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:15.796444    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:26:16.805549 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:16.805509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" event={"ID":"d2da9694-a921-472e-8392-74b043696b9d","Type":"ContainerStarted","Data":"e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51"}
Jul 27 22:26:17.785796 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:17.785763    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:26:17.786001 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:17.785957    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 27 22:26:17.786085 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:17.786011    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 27 22:26:18.786256 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:18.786219    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 27 22:26:18.786759 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:18.786283    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 27 22:26:19.786681 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:19.786640    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 27 22:26:19.787148 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:19.786709    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 27 22:26:20.786187 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:20.786153    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 27 22:26:20.786445 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:20.786209    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 27 22:26:21.790973 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:21.790944    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:26:21.791324 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:21.791213    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:26:21.791878 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:21.791861    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:26:30.451414 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.451380    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"]
Jul 27 22:26:30.451887 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.451628    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh" podUID="70a873c7-e272-486c-88c2-460eb727ea40" containerName="maas-api" containerID="cri-o://97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933" gracePeriod=30
Jul 27 22:26:30.721971 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.721947    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:30.808502 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.808458    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76hrw\" (UniqueName: \"kubernetes.io/projected/70a873c7-e272-486c-88c2-460eb727ea40-kube-api-access-76hrw\") pod \"70a873c7-e272-486c-88c2-460eb727ea40\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") "
Jul 27 22:26:30.808694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.808545    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a873c7-e272-486c-88c2-460eb727ea40-openshift-service-ca\") pod \"70a873c7-e272-486c-88c2-460eb727ea40\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") "
Jul 27 22:26:30.808694 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.808575    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls\") pod \"70a873c7-e272-486c-88c2-460eb727ea40\" (UID: \"70a873c7-e272-486c-88c2-460eb727ea40\") "
Jul 27 22:26:30.809003 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.808976    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70a873c7-e272-486c-88c2-460eb727ea40-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70a873c7-e272-486c-88c2-460eb727ea40" (UID: "70a873c7-e272-486c-88c2-460eb727ea40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:26:30.810796 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.810764    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70a873c7-e272-486c-88c2-460eb727ea40-kube-api-access-76hrw" (OuterVolumeSpecName: "kube-api-access-76hrw") pod "70a873c7-e272-486c-88c2-460eb727ea40" (UID: "70a873c7-e272-486c-88c2-460eb727ea40"). InnerVolumeSpecName "kube-api-access-76hrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:26:30.810894 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.810809    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70a873c7-e272-486c-88c2-460eb727ea40" (UID: "70a873c7-e272-486c-88c2-460eb727ea40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:26:30.885655 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.885618    2578 generic.go:356] "Generic (PLEG): container finished" podID="70a873c7-e272-486c-88c2-460eb727ea40" containerID="97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933" exitCode=0
Jul 27 22:26:30.885853 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.885663    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"
Jul 27 22:26:30.885853 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.885705    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh" event={"ID":"70a873c7-e272-486c-88c2-460eb727ea40","Type":"ContainerDied","Data":"97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933"}
Jul 27 22:26:30.885853 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.885777    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh" event={"ID":"70a873c7-e272-486c-88c2-460eb727ea40","Type":"ContainerDied","Data":"905c6bbbe06c50be22631686f132d73ccfdc406cf3392380f2a24c9e5bb44587"}
Jul 27 22:26:30.885853 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.885807    2578 scope.go:117] "RemoveContainer" containerID="97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933"
Jul 27 22:26:30.896475 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.896457    2578 scope.go:117] "RemoveContainer" containerID="97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933"
Jul 27 22:26:30.896780 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:26:30.896762    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933\": container with ID starting with 97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933 not found: ID does not exist" containerID="97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933"
Jul 27 22:26:30.896836 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.896790    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933"} err="failed to get container status \"97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933\": rpc error: code = NotFound desc = could not find container \"97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933\": container with ID starting with 97960e66b6ef0476782fcfb0aead399606899d598c5283d81b1cfa85d0641933 not found: ID does not exist"
Jul 27 22:26:30.909560 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.909539    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76hrw\" (UniqueName: \"kubernetes.io/projected/70a873c7-e272-486c-88c2-460eb727ea40-kube-api-access-76hrw\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:30.909560 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.909559    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a873c7-e272-486c-88c2-460eb727ea40-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:30.909671 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.909569    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a873c7-e272-486c-88c2-460eb727ea40-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:30.931153 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.931129    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"]
Jul 27 22:26:30.934646 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:30.934626    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eab778-585ff7fb86-979jh"]
Jul 27 22:26:31.606797 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:31.606763    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70a873c7-e272-486c-88c2-460eb727ea40" path="/var/lib/kubelet/pods/70a873c7-e272-486c-88c2-460eb727ea40/volumes"
Jul 27 22:26:38.433604 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:38.433567    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"]
Jul 27 22:26:38.433991 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:38.433826    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy" containerID="cri-o://e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51" gracePeriod=30
Jul 27 22:26:43.700516 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.700464    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:26:43.820866 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.820841    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-data\") pod \"d2da9694-a921-472e-8392-74b043696b9d\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") "
Jul 27 22:26:43.821048 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.820877    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-certs\") pod \"d2da9694-a921-472e-8392-74b043696b9d\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") "
Jul 27 22:26:43.821048 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.820921    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2da9694-a921-472e-8392-74b043696b9d-istio-podinfo\") pod \"d2da9694-a921-472e-8392-74b043696b9d\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") "
Jul 27 22:26:43.821048 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.820947    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blznk\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-kube-api-access-blznk\") pod \"d2da9694-a921-472e-8392-74b043696b9d\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") "
Jul 27 22:26:43.821048 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.820974    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-envoy\") pod \"d2da9694-a921-472e-8392-74b043696b9d\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") "
Jul 27 22:26:43.821048 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821029    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-istio-token\") pod \"d2da9694-a921-472e-8392-74b043696b9d\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") "
Jul 27 22:26:43.821350 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821054    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-socket\") pod \"d2da9694-a921-472e-8392-74b043696b9d\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") "
Jul 27 22:26:43.821350 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821074    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-credential-socket\") pod \"d2da9694-a921-472e-8392-74b043696b9d\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") "
Jul 27 22:26:43.821350 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821111    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2da9694-a921-472e-8392-74b043696b9d-istiod-ca-cert\") pod \"d2da9694-a921-472e-8392-74b043696b9d\" (UID: \"d2da9694-a921-472e-8392-74b043696b9d\") "
Jul 27 22:26:43.821350 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821161    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d2da9694-a921-472e-8392-74b043696b9d" (UID: "d2da9694-a921-472e-8392-74b043696b9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:26:43.821562 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821384    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d2da9694-a921-472e-8392-74b043696b9d" (UID: "d2da9694-a921-472e-8392-74b043696b9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:26:43.821562 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821461    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2da9694-a921-472e-8392-74b043696b9d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d2da9694-a921-472e-8392-74b043696b9d" (UID: "d2da9694-a921-472e-8392-74b043696b9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:26:43.821668 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821642    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:43.821721 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821661    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:43.821721 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821693    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d2da9694-a921-472e-8392-74b043696b9d" (UID: "d2da9694-a921-472e-8392-74b043696b9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:26:43.821875 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821703    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2da9694-a921-472e-8392-74b043696b9d-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:43.821875 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.821829    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-data" (OuterVolumeSpecName: "istio-data") pod "d2da9694-a921-472e-8392-74b043696b9d" (UID: "d2da9694-a921-472e-8392-74b043696b9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:26:43.823719 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.823693    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-kube-api-access-blznk" (OuterVolumeSpecName: "kube-api-access-blznk") pod "d2da9694-a921-472e-8392-74b043696b9d" (UID: "d2da9694-a921-472e-8392-74b043696b9d"). InnerVolumeSpecName "kube-api-access-blznk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:26:43.823908 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.823764    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-istio-token" (OuterVolumeSpecName: "istio-token") pod "d2da9694-a921-472e-8392-74b043696b9d" (UID: "d2da9694-a921-472e-8392-74b043696b9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:26:43.824055 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.824033    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d2da9694-a921-472e-8392-74b043696b9d" (UID: "d2da9694-a921-472e-8392-74b043696b9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:26:43.824119 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.824103    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2da9694-a921-472e-8392-74b043696b9d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d2da9694-a921-472e-8392-74b043696b9d" (UID: "d2da9694-a921-472e-8392-74b043696b9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:26:43.922631 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.922593    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:43.922631 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.922626    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2da9694-a921-472e-8392-74b043696b9d-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:43.922631 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.922636    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blznk\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-kube-api-access-blznk\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:43.922860 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.922645    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:43.922860 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.922653    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2da9694-a921-472e-8392-74b043696b9d-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:43.922860 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.922660    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2da9694-a921-472e-8392-74b043696b9d-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:43.954672 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.954604    2578 generic.go:356] "Generic (PLEG): container finished" podID="d2da9694-a921-472e-8392-74b043696b9d" containerID="e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51" exitCode=0
Jul 27 22:26:43.954672 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.954651    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"
Jul 27 22:26:43.954898 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.954690    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" event={"ID":"d2da9694-a921-472e-8392-74b043696b9d","Type":"ContainerDied","Data":"e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51"}
Jul 27 22:26:43.954898 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.954774    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l" event={"ID":"d2da9694-a921-472e-8392-74b043696b9d","Type":"ContainerDied","Data":"a21207d11056e981a9cc1b907a273ebad0b5b4569cc8836967f7168abfdfa67f"}
Jul 27 22:26:43.954898 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.954808    2578 scope.go:117] "RemoveContainer" containerID="e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51"
Jul 27 22:26:43.964697 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.964682    2578 scope.go:117] "RemoveContainer" containerID="67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251"
Jul 27 22:26:43.973012 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.972995    2578 scope.go:117] "RemoveContainer" containerID="e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51"
Jul 27 22:26:43.973258 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:26:43.973237    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51\": container with ID starting with e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51 not found: ID does not exist" containerID="e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51"
Jul 27 22:26:43.973330 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.973272    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51"} err="failed to get container status \"e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51\": rpc error: code = NotFound desc = could not find container \"e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51\": container with ID starting with e594c246987e1e5ac217c9775d85500b7bd97c7c7b12aacca8d8fcddc88d7e51 not found: ID does not exist"
Jul 27 22:26:43.973330 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.973297    2578 scope.go:117] "RemoveContainer" containerID="67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251"
Jul 27 22:26:43.973530 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:26:43.973512    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251\": container with ID starting with 67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251 not found: ID does not exist" containerID="67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251"
Jul 27 22:26:43.973578 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.973536    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251"} err="failed to get container status \"67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251\": rpc error: code = NotFound desc = could not find container \"67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251\": container with ID starting with 67f0d82a08d93430e592a5adf653b41d282898c20619c8bb4a09d4f3cd84a251 not found: ID does not exist"
Jul 27 22:26:43.978880 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.978859    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"]
Jul 27 22:26:43.983808 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:43.983786    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eab778-openshift-default-6b4d969dfb-hg59l"]
Jul 27 22:26:45.605518 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:45.605481    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2da9694-a921-472e-8392-74b043696b9d" path="/var/lib/kubelet/pods/d2da9694-a921-472e-8392-74b043696b9d/volumes"
Jul 27 22:26:57.261205 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.261169    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"]
Jul 27 22:26:57.261719 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.261448    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb" podUID="e1182234-1160-4b2f-80e3-07c3f6cc3af6" containerName="maas-api" containerID="cri-o://11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8" gracePeriod=30
Jul 27 22:26:57.536318 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.536290    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:26:57.646877 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.646848    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld94z\" (UniqueName: \"kubernetes.io/projected/e1182234-1160-4b2f-80e3-07c3f6cc3af6-kube-api-access-ld94z\") pod \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") "
Jul 27 22:26:57.646877 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.646879    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1182234-1160-4b2f-80e3-07c3f6cc3af6-maas-api-tls\") pod \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") "
Jul 27 22:26:57.647065 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.646919    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1182234-1160-4b2f-80e3-07c3f6cc3af6-openshift-service-ca\") pod \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\" (UID: \"e1182234-1160-4b2f-80e3-07c3f6cc3af6\") "
Jul 27 22:26:57.647325 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.647297    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1182234-1160-4b2f-80e3-07c3f6cc3af6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e1182234-1160-4b2f-80e3-07c3f6cc3af6" (UID: "e1182234-1160-4b2f-80e3-07c3f6cc3af6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:26:57.649189 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.649169    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1182234-1160-4b2f-80e3-07c3f6cc3af6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e1182234-1160-4b2f-80e3-07c3f6cc3af6" (UID: "e1182234-1160-4b2f-80e3-07c3f6cc3af6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:26:57.649294 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.649270    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1182234-1160-4b2f-80e3-07c3f6cc3af6-kube-api-access-ld94z" (OuterVolumeSpecName: "kube-api-access-ld94z") pod "e1182234-1160-4b2f-80e3-07c3f6cc3af6" (UID: "e1182234-1160-4b2f-80e3-07c3f6cc3af6"). InnerVolumeSpecName "kube-api-access-ld94z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:26:57.748282 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.748250    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld94z\" (UniqueName: \"kubernetes.io/projected/e1182234-1160-4b2f-80e3-07c3f6cc3af6-kube-api-access-ld94z\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:57.748454 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.748388    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1182234-1160-4b2f-80e3-07c3f6cc3af6-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:57.748454 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:57.748411    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1182234-1160-4b2f-80e3-07c3f6cc3af6-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:26:58.029606 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:58.029572    2578 generic.go:356] "Generic (PLEG): container finished" podID="e1182234-1160-4b2f-80e3-07c3f6cc3af6" containerID="11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8" exitCode=0
Jul 27 22:26:58.029795 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:58.029623    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"
Jul 27 22:26:58.029795 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:58.029662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb" event={"ID":"e1182234-1160-4b2f-80e3-07c3f6cc3af6","Type":"ContainerDied","Data":"11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8"}
Jul 27 22:26:58.029795 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:58.029715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb" event={"ID":"e1182234-1160-4b2f-80e3-07c3f6cc3af6","Type":"ContainerDied","Data":"eaa64f54da83d5f5bf1fd4de37022a926573961dc0ff9f5b0c551282d2ed5a3d"}
Jul 27 22:26:58.029795 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:58.029764    2578 scope.go:117] "RemoveContainer" containerID="11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8"
Jul 27 22:26:58.039830 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:58.039808    2578 scope.go:117] "RemoveContainer" containerID="11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8"
Jul 27 22:26:58.040086 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:26:58.040066    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8\": container with ID starting with 11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8 not found: ID does not exist" containerID="11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8"
Jul 27 22:26:58.040137 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:58.040096    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8"} err="failed to get container status \"11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8\": rpc error: code = NotFound desc = could not find container \"11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8\": container with ID starting with 11d8840de19cd190142591a38b840d0143d5ce51b008e2f0308324a28a212ec8 not found: ID does not exist"
Jul 27 22:26:58.052653 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:58.052631    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"]
Jul 27 22:26:58.058889 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:58.058866    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dd2178-8d59889f5-b9ppb"]
Jul 27 22:26:59.605586 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:26:59.605552    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1182234-1160-4b2f-80e3-07c3f6cc3af6" path="/var/lib/kubelet/pods/e1182234-1160-4b2f-80e3-07c3f6cc3af6/volumes"
Jul 27 22:28:12.643946 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:12.643915    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"]
Jul 27 22:28:12.644519 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:12.644120    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k" podUID="a6d45b6f-f05c-48b2-bc62-9e2612945f97" containerName="maas-api" containerID="cri-o://36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf" gracePeriod=30
Jul 27 22:28:12.900347 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:12.900285    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:28:12.955004 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:12.954965    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6d45b6f-f05c-48b2-bc62-9e2612945f97-maas-api-tls\") pod \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") "
Jul 27 22:28:12.955185 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:12.955048    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-727bx\" (UniqueName: \"kubernetes.io/projected/a6d45b6f-f05c-48b2-bc62-9e2612945f97-kube-api-access-727bx\") pod \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") "
Jul 27 22:28:12.955185 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:12.955091    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d45b6f-f05c-48b2-bc62-9e2612945f97-openshift-service-ca\") pod \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\" (UID: \"a6d45b6f-f05c-48b2-bc62-9e2612945f97\") "
Jul 27 22:28:12.955546 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:12.955511    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6d45b6f-f05c-48b2-bc62-9e2612945f97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a6d45b6f-f05c-48b2-bc62-9e2612945f97" (UID: "a6d45b6f-f05c-48b2-bc62-9e2612945f97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:28:12.957422 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:12.957400    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6d45b6f-f05c-48b2-bc62-9e2612945f97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a6d45b6f-f05c-48b2-bc62-9e2612945f97" (UID: "a6d45b6f-f05c-48b2-bc62-9e2612945f97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:28:12.957561 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:12.957534    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6d45b6f-f05c-48b2-bc62-9e2612945f97-kube-api-access-727bx" (OuterVolumeSpecName: "kube-api-access-727bx") pod "a6d45b6f-f05c-48b2-bc62-9e2612945f97" (UID: "a6d45b6f-f05c-48b2-bc62-9e2612945f97"). InnerVolumeSpecName "kube-api-access-727bx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:13.056486 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.056458    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d45b6f-f05c-48b2-bc62-9e2612945f97-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:13.056486 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.056488    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6d45b6f-f05c-48b2-bc62-9e2612945f97-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:13.056669 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.056503    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-727bx\" (UniqueName: \"kubernetes.io/projected/a6d45b6f-f05c-48b2-bc62-9e2612945f97-kube-api-access-727bx\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:13.417510 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.417471    2578 generic.go:356] "Generic (PLEG): container finished" podID="a6d45b6f-f05c-48b2-bc62-9e2612945f97" containerID="36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf" exitCode=0
Jul 27 22:28:13.417695 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.417523    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"
Jul 27 22:28:13.417695 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.417522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k" event={"ID":"a6d45b6f-f05c-48b2-bc62-9e2612945f97","Type":"ContainerDied","Data":"36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf"}
Jul 27 22:28:13.417695 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.417675    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k" event={"ID":"a6d45b6f-f05c-48b2-bc62-9e2612945f97","Type":"ContainerDied","Data":"df659e2723b33717ae654bc0842ca9e0a6e7ee1dced97e661b9b1fa75bfb709b"}
Jul 27 22:28:13.417859 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.417726    2578 scope.go:117] "RemoveContainer" containerID="36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf"
Jul 27 22:28:13.427537 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.427519    2578 scope.go:117] "RemoveContainer" containerID="36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf"
Jul 27 22:28:13.427811 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:28:13.427787    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf\": container with ID starting with 36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf not found: ID does not exist" containerID="36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf"
Jul 27 22:28:13.427913 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.427813    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf"} err="failed to get container status \"36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf\": rpc error: code = NotFound desc = could not find container \"36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf\": container with ID starting with 36509a8f39285028331a41188d62c8731cbc76081cd4620f10b2b705f31e8bbf not found: ID does not exist"
Jul 27 22:28:13.450127 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.450102    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"]
Jul 27 22:28:13.464852 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.464828    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-54963f-659df8457-spn4k"]
Jul 27 22:28:13.606479 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:13.606445    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6d45b6f-f05c-48b2-bc62-9e2612945f97" path="/var/lib/kubelet/pods/a6d45b6f-f05c-48b2-bc62-9e2612945f97/volumes"
Jul 27 22:28:27.681449 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:27.681408    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"]
Jul 27 22:28:27.682590 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:27.682532    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc" podUID="af12b272-3cff-4da8-b157-88f822e97792" containerName="maas-api" containerID="cri-o://25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32" gracePeriod=30
Jul 27 22:28:27.935789 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:27.935711    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:28:28.100631 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.100597    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af12b272-3cff-4da8-b157-88f822e97792-maas-api-tls\") pod \"af12b272-3cff-4da8-b157-88f822e97792\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") "
Jul 27 22:28:28.100631 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.100635    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7lbc\" (UniqueName: \"kubernetes.io/projected/af12b272-3cff-4da8-b157-88f822e97792-kube-api-access-f7lbc\") pod \"af12b272-3cff-4da8-b157-88f822e97792\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") "
Jul 27 22:28:28.100929 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.100662    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af12b272-3cff-4da8-b157-88f822e97792-openshift-service-ca\") pod \"af12b272-3cff-4da8-b157-88f822e97792\" (UID: \"af12b272-3cff-4da8-b157-88f822e97792\") "
Jul 27 22:28:28.101128 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.101088    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af12b272-3cff-4da8-b157-88f822e97792-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af12b272-3cff-4da8-b157-88f822e97792" (UID: "af12b272-3cff-4da8-b157-88f822e97792"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:28:28.103101 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.103060    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af12b272-3cff-4da8-b157-88f822e97792-kube-api-access-f7lbc" (OuterVolumeSpecName: "kube-api-access-f7lbc") pod "af12b272-3cff-4da8-b157-88f822e97792" (UID: "af12b272-3cff-4da8-b157-88f822e97792"). InnerVolumeSpecName "kube-api-access-f7lbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:28.103200 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.103096    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af12b272-3cff-4da8-b157-88f822e97792-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af12b272-3cff-4da8-b157-88f822e97792" (UID: "af12b272-3cff-4da8-b157-88f822e97792"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:28:28.202291 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.202231    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af12b272-3cff-4da8-b157-88f822e97792-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:28.202291 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.202252    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7lbc\" (UniqueName: \"kubernetes.io/projected/af12b272-3cff-4da8-b157-88f822e97792-kube-api-access-f7lbc\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:28.202291 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.202261    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af12b272-3cff-4da8-b157-88f822e97792-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:28.494791 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.494703    2578 generic.go:356] "Generic (PLEG): container finished" podID="af12b272-3cff-4da8-b157-88f822e97792" containerID="25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32" exitCode=0
Jul 27 22:28:28.494791 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.494767    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc" event={"ID":"af12b272-3cff-4da8-b157-88f822e97792","Type":"ContainerDied","Data":"25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32"}
Jul 27 22:28:28.494791 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.494774    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"
Jul 27 22:28:28.494999 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.494817    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc" event={"ID":"af12b272-3cff-4da8-b157-88f822e97792","Type":"ContainerDied","Data":"24286a085b82d44d6a624fe99ce62f886ca02967f4acd9539bf5b8fba73354fa"}
Jul 27 22:28:28.494999 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.494841    2578 scope.go:117] "RemoveContainer" containerID="25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32"
Jul 27 22:28:28.504682 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.504663    2578 scope.go:117] "RemoveContainer" containerID="25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32"
Jul 27 22:28:28.504971 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:28:28.504949    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32\": container with ID starting with 25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32 not found: ID does not exist" containerID="25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32"
Jul 27 22:28:28.505031 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.504979    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32"} err="failed to get container status \"25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32\": rpc error: code = NotFound desc = could not find container \"25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32\": container with ID starting with 25564e822c01e3595f62ee3d8f591e6a2f8c549733f79009d430431809710d32 not found: ID does not exist"
Jul 27 22:28:28.521151 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.521120    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"]
Jul 27 22:28:28.530621 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:28.530589    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-94450e-6996b76556-qvrmc"]
Jul 27 22:28:29.605621 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:29.605581    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af12b272-3cff-4da8-b157-88f822e97792" path="/var/lib/kubelet/pods/af12b272-3cff-4da8-b157-88f822e97792/volumes"
Jul 27 22:28:35.560626 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:35.560595    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"]
Jul 27 22:28:35.561189 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:35.560886    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" podUID="978e5de1-fc85-4157-a53c-213d765a770e" containerName="istio-proxy" containerID="cri-o://4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda" gracePeriod=30
Jul 27 22:28:40.809055 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.809027    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:28:40.910567 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910538    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-certs\") pod \"978e5de1-fc85-4157-a53c-213d765a770e\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") "
Jul 27 22:28:40.910752 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910572    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-socket\") pod \"978e5de1-fc85-4157-a53c-213d765a770e\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") "
Jul 27 22:28:40.910752 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910650    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzwzr\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-kube-api-access-dzwzr\") pod \"978e5de1-fc85-4157-a53c-213d765a770e\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") "
Jul 27 22:28:40.910752 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910702    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-credential-socket\") pod \"978e5de1-fc85-4157-a53c-213d765a770e\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") "
Jul 27 22:28:40.910752 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910727    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-envoy\") pod \"978e5de1-fc85-4157-a53c-213d765a770e\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") "
Jul 27 22:28:40.910998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910787    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-istio-token\") pod \"978e5de1-fc85-4157-a53c-213d765a770e\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") "
Jul 27 22:28:40.910998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910837    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/978e5de1-fc85-4157-a53c-213d765a770e-istio-podinfo\") pod \"978e5de1-fc85-4157-a53c-213d765a770e\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") "
Jul 27 22:28:40.910998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910860    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/978e5de1-fc85-4157-a53c-213d765a770e-istiod-ca-cert\") pod \"978e5de1-fc85-4157-a53c-213d765a770e\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") "
Jul 27 22:28:40.910998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910858    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "978e5de1-fc85-4157-a53c-213d765a770e" (UID: "978e5de1-fc85-4157-a53c-213d765a770e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:40.910998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910890    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "978e5de1-fc85-4157-a53c-213d765a770e" (UID: "978e5de1-fc85-4157-a53c-213d765a770e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:40.910998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910930    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-data\") pod \"978e5de1-fc85-4157-a53c-213d765a770e\" (UID: \"978e5de1-fc85-4157-a53c-213d765a770e\") "
Jul 27 22:28:40.910998 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.910977    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "978e5de1-fc85-4157-a53c-213d765a770e" (UID: "978e5de1-fc85-4157-a53c-213d765a770e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:40.911357 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.911211    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:40.911357 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.911231    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:40.911357 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.911244    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:40.911357 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.911299    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/978e5de1-fc85-4157-a53c-213d765a770e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "978e5de1-fc85-4157-a53c-213d765a770e" (UID: "978e5de1-fc85-4157-a53c-213d765a770e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:28:40.911836 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.911811    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-data" (OuterVolumeSpecName: "istio-data") pod "978e5de1-fc85-4157-a53c-213d765a770e" (UID: "978e5de1-fc85-4157-a53c-213d765a770e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:40.913208 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.913179    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-kube-api-access-dzwzr" (OuterVolumeSpecName: "kube-api-access-dzwzr") pod "978e5de1-fc85-4157-a53c-213d765a770e" (UID: "978e5de1-fc85-4157-a53c-213d765a770e"). InnerVolumeSpecName "kube-api-access-dzwzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:40.913381 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.913362    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/978e5de1-fc85-4157-a53c-213d765a770e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "978e5de1-fc85-4157-a53c-213d765a770e" (UID: "978e5de1-fc85-4157-a53c-213d765a770e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:28:40.913381 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.913368    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-istio-token" (OuterVolumeSpecName: "istio-token") pod "978e5de1-fc85-4157-a53c-213d765a770e" (UID: "978e5de1-fc85-4157-a53c-213d765a770e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:40.913504 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:40.913421    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "978e5de1-fc85-4157-a53c-213d765a770e" (UID: "978e5de1-fc85-4157-a53c-213d765a770e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:41.011654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.011631    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzwzr\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-kube-api-access-dzwzr\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:41.011654 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.011653    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:41.011906 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.011662    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/978e5de1-fc85-4157-a53c-213d765a770e-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:41.011906 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.011670    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/978e5de1-fc85-4157-a53c-213d765a770e-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:41.011906 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.011677    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/978e5de1-fc85-4157-a53c-213d765a770e-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:41.011906 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.011685    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/978e5de1-fc85-4157-a53c-213d765a770e-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:41.159221 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.159176    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"]
Jul 27 22:28:41.159692 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.159654    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7" podUID="aba0ed7a-896c-48d2-a79c-71da71d534ae" containerName="maas-api" containerID="cri-o://3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c" gracePeriod=30
Jul 27 22:28:41.402127 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.402097    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:28:41.517066 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.516990    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5dks\" (UniqueName: \"kubernetes.io/projected/aba0ed7a-896c-48d2-a79c-71da71d534ae-kube-api-access-m5dks\") pod \"aba0ed7a-896c-48d2-a79c-71da71d534ae\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") "
Jul 27 22:28:41.517066 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.517046    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aba0ed7a-896c-48d2-a79c-71da71d534ae-openshift-service-ca\") pod \"aba0ed7a-896c-48d2-a79c-71da71d534ae\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") "
Jul 27 22:28:41.517066 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.517068    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aba0ed7a-896c-48d2-a79c-71da71d534ae-maas-api-tls\") pod \"aba0ed7a-896c-48d2-a79c-71da71d534ae\" (UID: \"aba0ed7a-896c-48d2-a79c-71da71d534ae\") "
Jul 27 22:28:41.517465 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.517415    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aba0ed7a-896c-48d2-a79c-71da71d534ae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aba0ed7a-896c-48d2-a79c-71da71d534ae" (UID: "aba0ed7a-896c-48d2-a79c-71da71d534ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:28:41.519318 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.519296    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aba0ed7a-896c-48d2-a79c-71da71d534ae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aba0ed7a-896c-48d2-a79c-71da71d534ae" (UID: "aba0ed7a-896c-48d2-a79c-71da71d534ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:28:41.519448 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.519339    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aba0ed7a-896c-48d2-a79c-71da71d534ae-kube-api-access-m5dks" (OuterVolumeSpecName: "kube-api-access-m5dks") pod "aba0ed7a-896c-48d2-a79c-71da71d534ae" (UID: "aba0ed7a-896c-48d2-a79c-71da71d534ae"). InnerVolumeSpecName "kube-api-access-m5dks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:41.566055 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.566026    2578 generic.go:356] "Generic (PLEG): container finished" podID="978e5de1-fc85-4157-a53c-213d765a770e" containerID="4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda" exitCode=0
Jul 27 22:28:41.566208 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.566079    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"
Jul 27 22:28:41.566208 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.566106    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" event={"ID":"978e5de1-fc85-4157-a53c-213d765a770e","Type":"ContainerDied","Data":"4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda"}
Jul 27 22:28:41.566208 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.566162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w" event={"ID":"978e5de1-fc85-4157-a53c-213d765a770e","Type":"ContainerDied","Data":"436e8866f1c7a944c2db6849ee955e890c2d0c9e27de5e944b84106daa7f86e7"}
Jul 27 22:28:41.566208 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.566188    2578 scope.go:117] "RemoveContainer" containerID="4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda"
Jul 27 22:28:41.567590 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.567568    2578 generic.go:356] "Generic (PLEG): container finished" podID="aba0ed7a-896c-48d2-a79c-71da71d534ae" containerID="3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c" exitCode=0
Jul 27 22:28:41.567703 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.567612    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7" event={"ID":"aba0ed7a-896c-48d2-a79c-71da71d534ae","Type":"ContainerDied","Data":"3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c"}
Jul 27 22:28:41.567703 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.567648    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7" event={"ID":"aba0ed7a-896c-48d2-a79c-71da71d534ae","Type":"ContainerDied","Data":"ea264b911a28cdb90799e9cf58c2d94328ef060f6ba4db1f99b30b53d35a4254"}
Jul 27 22:28:41.567703 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.567617    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"
Jul 27 22:28:41.577786 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.577765    2578 scope.go:117] "RemoveContainer" containerID="9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d"
Jul 27 22:28:41.591840 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.591818    2578 scope.go:117] "RemoveContainer" containerID="4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda"
Jul 27 22:28:41.592201 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:28:41.592177    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda\": container with ID starting with 4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda not found: ID does not exist" containerID="4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda"
Jul 27 22:28:41.592285 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.592210    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda"} err="failed to get container status \"4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda\": rpc error: code = NotFound desc = could not find container \"4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda\": container with ID starting with 4987e10ccbacd349c93b89a63b9b1b8d2dbaf8b467b4a5e0ba36f11c62fbefda not found: ID does not exist"
Jul 27 22:28:41.592285 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.592230    2578 scope.go:117] "RemoveContainer" containerID="9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d"
Jul 27 22:28:41.592549 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:28:41.592525    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d\": container with ID starting with 9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d not found: ID does not exist" containerID="9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d"
Jul 27 22:28:41.592598 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.592562    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d"} err="failed to get container status \"9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d\": rpc error: code = NotFound desc = could not find container \"9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d\": container with ID starting with 9551bbe4ed0800005236b720ed509c76806fbe82359184a670e8a2bea1fc663d not found: ID does not exist"
Jul 27 22:28:41.592598 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.592585    2578 scope.go:117] "RemoveContainer" containerID="3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c"
Jul 27 22:28:41.600805 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.600770    2578 scope.go:117] "RemoveContainer" containerID="3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c"
Jul 27 22:28:41.601037 ip-10-0-132-161 kubenswrapper[2578]: E0727 22:28:41.601018    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c\": container with ID starting with 3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c not found: ID does not exist" containerID="3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c"
Jul 27 22:28:41.601117 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.601043    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c"} err="failed to get container status \"3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c\": rpc error: code = NotFound desc = could not find container \"3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c\": container with ID starting with 3e8ad94a76ceea4ca9c8bb7c839c58c46f56983d5407395b78b7d4341837ed1c not found: ID does not exist"
Jul 27 22:28:41.606757 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.606711    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"]
Jul 27 22:28:41.606845 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.606806    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-94450e-openshift-default-6b68dcc9bd-rc45w"]
Jul 27 22:28:41.618224 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.618208    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aba0ed7a-896c-48d2-a79c-71da71d534ae-openshift-service-ca\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:41.618315 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.618226    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aba0ed7a-896c-48d2-a79c-71da71d534ae-maas-api-tls\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:41.618315 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.618236    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5dks\" (UniqueName: \"kubernetes.io/projected/aba0ed7a-896c-48d2-a79c-71da71d534ae-kube-api-access-m5dks\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:41.621619 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.621598    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"]
Jul 27 22:28:41.623374 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:41.623354    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0cb892-7d578bc69d-8qsz7"]
Jul 27 22:28:43.612040 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:43.612004    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="978e5de1-fc85-4157-a53c-213d765a770e" path="/var/lib/kubelet/pods/978e5de1-fc85-4157-a53c-213d765a770e/volumes"
Jul 27 22:28:43.612572 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:43.612551    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aba0ed7a-896c-48d2-a79c-71da71d534ae" path="/var/lib/kubelet/pods/aba0ed7a-896c-48d2-a79c-71da71d534ae/volumes"
Jul 27 22:28:48.505107 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:48.505075    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"]
Jul 27 22:28:48.505508 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:48.505319    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" podUID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerName="istio-proxy" containerID="cri-o://760ce2c5b8026a7fbaafbf92df40d44a783543110b10c7ecca47e8ffc8c72366" gracePeriod=30
Jul 27 22:28:49.759728 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:49.759703    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:28:49.779869 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:49.779843    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:28:53.629792 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.629763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" event={"ID":"e835e6a0-575a-42cd-93f3-aa5f86aa67c7","Type":"ContainerDied","Data":"760ce2c5b8026a7fbaafbf92df40d44a783543110b10c7ecca47e8ffc8c72366"}
Jul 27 22:28:53.630131 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.629726    2578 generic.go:356] "Generic (PLEG): container finished" podID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerID="760ce2c5b8026a7fbaafbf92df40d44a783543110b10c7ecca47e8ffc8c72366" exitCode=0
Jul 27 22:28:53.630131 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.629833    2578 scope.go:117] "RemoveContainer" containerID="dd5e7433c8d734b63ad56ef2ff7a98b081ce8246166e0bced69f682e6cc489c7"
Jul 27 22:28:53.762216 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.762193    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:28:53.817234 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817163    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-certs\") pod \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") "
Jul 27 22:28:53.817234 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817203    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-token\") pod \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") "
Jul 27 22:28:53.817234 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817231    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-credential-socket\") pod \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") "
Jul 27 22:28:53.817509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817247    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-data\") pod \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") "
Jul 27 22:28:53.817509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817273    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-socket\") pod \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") "
Jul 27 22:28:53.817509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817330    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istiod-ca-cert\") pod \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") "
Jul 27 22:28:53.817509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817347    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-envoy\") pod \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") "
Jul 27 22:28:53.817509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817417    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88hjz\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-kube-api-access-88hjz\") pod \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") "
Jul 27 22:28:53.817509 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817451    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-podinfo\") pod \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\" (UID: \"e835e6a0-575a-42cd-93f3-aa5f86aa67c7\") "
Jul 27 22:28:53.817858 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817547    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e835e6a0-575a-42cd-93f3-aa5f86aa67c7" (UID: "e835e6a0-575a-42cd-93f3-aa5f86aa67c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:53.817858 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817683    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-credential-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:53.817858 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817726    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e835e6a0-575a-42cd-93f3-aa5f86aa67c7" (UID: "e835e6a0-575a-42cd-93f3-aa5f86aa67c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:53.818021 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.817929    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e835e6a0-575a-42cd-93f3-aa5f86aa67c7" (UID: "e835e6a0-575a-42cd-93f3-aa5f86aa67c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:53.818076 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.818027    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e835e6a0-575a-42cd-93f3-aa5f86aa67c7" (UID: "e835e6a0-575a-42cd-93f3-aa5f86aa67c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:28:53.818205 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.818176    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "e835e6a0-575a-42cd-93f3-aa5f86aa67c7" (UID: "e835e6a0-575a-42cd-93f3-aa5f86aa67c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:53.820237 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.820215    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e835e6a0-575a-42cd-93f3-aa5f86aa67c7" (UID: "e835e6a0-575a-42cd-93f3-aa5f86aa67c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:53.820363 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.820345    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e835e6a0-575a-42cd-93f3-aa5f86aa67c7" (UID: "e835e6a0-575a-42cd-93f3-aa5f86aa67c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 22:28:53.820432 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.820390    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "e835e6a0-575a-42cd-93f3-aa5f86aa67c7" (UID: "e835e6a0-575a-42cd-93f3-aa5f86aa67c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:53.820432 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.820401    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-kube-api-access-88hjz" (OuterVolumeSpecName: "kube-api-access-88hjz") pod "e835e6a0-575a-42cd-93f3-aa5f86aa67c7" (UID: "e835e6a0-575a-42cd-93f3-aa5f86aa67c7"). InnerVolumeSpecName "kube-api-access-88hjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:53.918942 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.918914    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88hjz\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-kube-api-access-88hjz\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:53.918942 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.918937    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-podinfo\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:53.918942 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.918946    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-certs\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:53.919160 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.918954    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-token\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:53.919160 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.918962    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-data\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:53.919160 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.918970    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-workload-socket\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:53.919160 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.918978    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istiod-ca-cert\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:53.919160 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:53.918987    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e835e6a0-575a-42cd-93f3-aa5f86aa67c7-istio-envoy\") on node \"ip-10-0-132-161.ec2.internal\" DevicePath \"\""
Jul 27 22:28:54.636351 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:54.636319    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78" event={"ID":"e835e6a0-575a-42cd-93f3-aa5f86aa67c7","Type":"ContainerDied","Data":"7fb30ef7d07a512a549092f98e2bb7dcbf30311fa287e475442c43027b026587"}
Jul 27 22:28:54.636832 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:54.636364    2578 scope.go:117] "RemoveContainer" containerID="760ce2c5b8026a7fbaafbf92df40d44a783543110b10c7ecca47e8ffc8c72366"
Jul 27 22:28:54.636832 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:54.636324    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"
Jul 27 22:28:54.660161 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:54.660138    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"]
Jul 27 22:28:54.662900 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:54.662880    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-0cb892-openshift-default-6fbcdf64-jkb78"]
Jul 27 22:28:54.978186 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:54.978166    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6fbcf7897d-j27pr_0460a571-45f1-4f8a-98a0-44744263005f/manager/0.log"
Jul 27 22:28:55.491029 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:55.491005    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d57c545b5-7pdmp_bed8d6e8-1317-4d45-913d-db49e3c2e28f/copy-manifests/0.log"
Jul 27 22:28:55.505095 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:55.505071    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d57c545b5-7pdmp_bed8d6e8-1317-4d45-913d-db49e3c2e28f/manager/0.log"
Jul 27 22:28:55.608631 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:55.608599    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" path="/var/lib/kubelet/pods/e835e6a0-575a-42cd-93f3-aa5f86aa67c7/volumes"
Jul 27 22:28:56.012749 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:56.012711    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-bc9f765f9-lm6bf_d8a613c4-92d9-4301-a365-f4df038b5b5b/maas-api/0.log"
Jul 27 22:28:56.124190 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:56.124159    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753145-wzml6_0b521213-1c52-4635-8277-981411b3f138/cleanup/0.log"
Jul 27 22:28:56.233035 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:56.233011    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753160-l92k8_cd1ef333-8aac-4d56-bf42-68e4bde1d487/cleanup/0.log"
Jul 27 22:28:56.460529 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:56.460501    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ptpgp_d5dc2039-640a-4377-8beb-fac18555ee1b/postgres/0.log"
Jul 27 22:28:59.116778 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:59.116749    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-l224d_971ff8b7-349e-4224-b9c5-95798d96ab1d/discovery/0.log"
Jul 27 22:28:59.708228 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:59.708197    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-885jp_05798666-dd9a-440c-98c8-280baf18deb3/payload-processing/0.log"
Jul 27 22:28:59.829746 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:28:59.829715    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64884fdbff-j7wtj_4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f/router/0.log"
Jul 27 22:29:00.312086 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:00.312061    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dkdkc_060c5768-72ca-4bd5-bcbe-935f60768fee/main/0.log"
Jul 27 22:29:00.427145 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:00.427124    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-rttgj_ef18c4b1-db1f-48d5-8dbf-0cb255c7465d/main/0.log"
Jul 27 22:29:09.252308 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:09.252276    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2q7dv_d3b83b76-20e7-43e7-85bd-885ab2b22ce8/global-pull-secret-syncer/0.log"
Jul 27 22:29:09.592324 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:09.592243    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q5jzr_24413672-bc61-4434-aa07-40013e6257a6/konnectivity-agent/0.log"
Jul 27 22:29:09.618599 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:09.618572    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-161.ec2.internal_19538ed6d308dc3f353d7b9a9b7dce11/haproxy/0.log"
Jul 27 22:29:17.243884 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:17.243833    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-g8gck_a56677e1-f2c8-4000-b957-9cb3d83fc8a1/cluster-monitoring-operator/0.log"
Jul 27 22:29:17.564482 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:17.564399    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cmwkj_a547c558-4bb1-4759-a3c9-611871d54566/node-exporter/0.log"
Jul 27 22:29:17.605556 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:17.605520    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cmwkj_a547c558-4bb1-4759-a3c9-611871d54566/kube-rbac-proxy/0.log"
Jul 27 22:29:17.641598 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:17.641579    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cmwkj_a547c558-4bb1-4759-a3c9-611871d54566/init-textfile/0.log"
Jul 27 22:29:19.612622 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.612597    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-dr8rd_0442a5bc-b902-4467-bfc2-508cc0aeab47/networking-console-plugin/0.log"
Jul 27 22:29:19.754948 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.754914    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"]
Jul 27 22:29:19.755250 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755239    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aba0ed7a-896c-48d2-a79c-71da71d534ae" containerName="maas-api"
Jul 27 22:29:19.755295 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755252    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aba0ed7a-896c-48d2-a79c-71da71d534ae" containerName="maas-api"
Jul 27 22:29:19.755295 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755261    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy"
Jul 27 22:29:19.755295 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755267    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy"
Jul 27 22:29:19.755295 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755278    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="978e5de1-fc85-4157-a53c-213d765a770e" containerName="istio-proxy"
Jul 27 22:29:19.755295 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755285    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="978e5de1-fc85-4157-a53c-213d765a770e" containerName="istio-proxy"
Jul 27 22:29:19.755295 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755292    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af12b272-3cff-4da8-b157-88f822e97792" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755298    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="af12b272-3cff-4da8-b157-88f822e97792" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755306    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerName="istio-proxy"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755311    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerName="istio-proxy"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755316    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755321    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755329    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1182234-1160-4b2f-80e3-07c3f6cc3af6" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755334    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1182234-1160-4b2f-80e3-07c3f6cc3af6" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755340    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70a873c7-e272-486c-88c2-460eb727ea40" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755344    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="70a873c7-e272-486c-88c2-460eb727ea40" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755351    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6d45b6f-f05c-48b2-bc62-9e2612945f97" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755355    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6d45b6f-f05c-48b2-bc62-9e2612945f97" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755418    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="aba0ed7a-896c-48d2-a79c-71da71d534ae" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755425    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="af12b272-3cff-4da8-b157-88f822e97792" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755431    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1182234-1160-4b2f-80e3-07c3f6cc3af6" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755437    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="978e5de1-fc85-4157-a53c-213d765a770e" containerName="istio-proxy"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755443    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755448    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="70a873c7-e272-486c-88c2-460eb727ea40" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755452    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e835e6a0-575a-42cd-93f3-aa5f86aa67c7" containerName="istio-proxy"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755458    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6d45b6f-f05c-48b2-bc62-9e2612945f97" containerName="maas-api"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755463    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="978e5de1-fc85-4157-a53c-213d765a770e" containerName="istio-proxy"
Jul 27 22:29:19.755477 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.755467    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2da9694-a921-472e-8392-74b043696b9d" containerName="istio-proxy"
Jul 27 22:29:19.758468 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.758445    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.761519 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.761484    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j99rk\"/\"openshift-service-ca.crt\""
Jul 27 22:29:19.762678 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.762658    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j99rk\"/\"default-dockercfg-f5phm\""
Jul 27 22:29:19.762816 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.762795    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j99rk\"/\"kube-root-ca.crt\""
Jul 27 22:29:19.774922 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.774896    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"]
Jul 27 22:29:19.827897 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.827864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-proc\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.827897 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.827895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-sys\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.828164 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.827925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xljdk\" (UniqueName: \"kubernetes.io/projected/af19362b-f777-494a-8f6a-fc64a52b513c-kube-api-access-xljdk\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.828164 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.828112    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-podres\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.828273 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.828161    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-lib-modules\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.928969 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.928930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-podres\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.928969 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.928973    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-lib-modules\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.929241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.928994    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-proc\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.929241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.929028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-sys\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.929241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.929055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xljdk\" (UniqueName: \"kubernetes.io/projected/af19362b-f777-494a-8f6a-fc64a52b513c-kube-api-access-xljdk\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.929241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.929092    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-proc\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.929241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.929101    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-podres\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.929241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.929130    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-lib-modules\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.929241 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.929150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af19362b-f777-494a-8f6a-fc64a52b513c-sys\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:19.939879 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:19.939851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xljdk\" (UniqueName: \"kubernetes.io/projected/af19362b-f777-494a-8f6a-fc64a52b513c-kube-api-access-xljdk\") pod \"perf-node-gather-daemonset-db8mz\" (UID: \"af19362b-f777-494a-8f6a-fc64a52b513c\") " pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:20.068782 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:20.068714    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:20.209924 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:20.209724    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"]
Jul 27 22:29:20.212152 ip-10-0-132-161 kubenswrapper[2578]: W0727 22:29:20.212105    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaf19362b_f777_494a_8f6a_fc64a52b513c.slice/crio-b04b5c42aa7c300c1a02e47510be5c9de35447990ac4aec92d9e321b75619cff WatchSource:0}: Error finding container b04b5c42aa7c300c1a02e47510be5c9de35447990ac4aec92d9e321b75619cff: Status 404 returned error can't find the container with id b04b5c42aa7c300c1a02e47510be5c9de35447990ac4aec92d9e321b75619cff
Jul 27 22:29:20.417457 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:20.417424    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/1.log"
Jul 27 22:29:20.422079 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:20.422057    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-wzbsj_22ef61cc-b722-40e9-ad7e-4819d634238f/console-operator/2.log"
Jul 27 22:29:20.750610 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:20.750572    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz" event={"ID":"af19362b-f777-494a-8f6a-fc64a52b513c","Type":"ContainerStarted","Data":"d06ecf4b6b17db7691642cadd0471be2f133f8c6b35536e668d54b9b4612076c"}
Jul 27 22:29:20.751064 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:20.750648    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz" event={"ID":"af19362b-f777-494a-8f6a-fc64a52b513c","Type":"ContainerStarted","Data":"b04b5c42aa7c300c1a02e47510be5c9de35447990ac4aec92d9e321b75619cff"}
Jul 27 22:29:20.751064 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:20.750750    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:20.769623 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:20.769573    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz" podStartSLOduration=1.769559361 podStartE2EDuration="1.769559361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 22:29:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:29:20.76840876 +0000 UTC m=+3331.664218212" watchObservedRunningTime="2026-07-27 22:29:20.769559361 +0000 UTC m=+3331.665368811"
Jul 27 22:29:20.960843 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:20.960809    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-6b4dx_503f8bfb-1452-4193-96c3-5d6dc36bc410/download-server/0.log"
Jul 27 22:29:21.567479 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:21.567450    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-vrdbh_34756d4e-1ce5-4140-8585-fae29e941ab3/volume-data-source-validator/0.log"
Jul 27 22:29:22.638672 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:22.638643    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qdxcv_2fd239d2-c68e-4edf-a18b-1e2888c14dcc/dns/0.log"
Jul 27 22:29:22.669050 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:22.669020    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qdxcv_2fd239d2-c68e-4edf-a18b-1e2888c14dcc/kube-rbac-proxy/0.log"
Jul 27 22:29:22.827578 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:22.827542    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ftzbq_c9f9f3fc-17d0-4b1d-8a50-01757a4f934c/dns-node-resolver/0.log"
Jul 27 22:29:23.505262 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:23.505223    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-22g6x_d32a5315-4ac2-4d35-b133-854854da98c7/node-ca/0.log"
Jul 27 22:29:24.699121 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:24.699091    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-l224d_971ff8b7-349e-4224-b9c5-95798d96ab1d/discovery/0.log"
Jul 27 22:29:24.907278 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:24.907249    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-885jp_05798666-dd9a-440c-98c8-280baf18deb3/payload-processing/0.log"
Jul 27 22:29:24.940987 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:24.940956    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64884fdbff-j7wtj_4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f/router/0.log"
Jul 27 22:29:25.777674 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:25.777647    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8xsl4_6d42297f-2c9c-4373-afc5-667b897f9c57/serve-healthcheck-canary/0.log"
Jul 27 22:29:26.347096 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:26.347062    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-cpgfx_3a511784-485f-4334-b266-e9e7dea58199/insights-operator/0.log"
Jul 27 22:29:26.348094 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:26.348076    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-cpgfx_3a511784-485f-4334-b266-e9e7dea58199/insights-operator/1.log"
Jul 27 22:29:26.464099 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:26.464070    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r5djc_eabb1707-799d-4ec3-a787-87acc0a7955b/kube-rbac-proxy/0.log"
Jul 27 22:29:26.488322 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:26.488296    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r5djc_eabb1707-799d-4ec3-a787-87acc0a7955b/exporter/0.log"
Jul 27 22:29:26.515136 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:26.515110    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r5djc_eabb1707-799d-4ec3-a787-87acc0a7955b/extractor/0.log"
Jul 27 22:29:26.766596 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:26.766564    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-j99rk/perf-node-gather-daemonset-db8mz"
Jul 27 22:29:29.783959 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:29.783924    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6fbcf7897d-j27pr_0460a571-45f1-4f8a-98a0-44744263005f/manager/0.log"
Jul 27 22:29:30.201593 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:30.201506    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d57c545b5-7pdmp_bed8d6e8-1317-4d45-913d-db49e3c2e28f/manager/0.log"
Jul 27 22:29:30.229807 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:30.229780    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d57c545b5-7pdmp_bed8d6e8-1317-4d45-913d-db49e3c2e28f/copy-manifests/0.log"
Jul 27 22:29:30.288895 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:30.288859    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-l224d_971ff8b7-349e-4224-b9c5-95798d96ab1d/discovery/0.log"
Jul 27 22:29:30.450512 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:30.450478    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-885jp_05798666-dd9a-440c-98c8-280baf18deb3/payload-processing/0.log"
Jul 27 22:29:30.463573 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:30.463500    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64884fdbff-j7wtj_4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f/router/0.log"
Jul 27 22:29:31.591144 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:31.591113    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7c57b59594-lvwjn_4f8f7fc9-f006-42bf-bbf1-ff6f9ecbec44/manager/0.log"
Jul 27 22:29:32.011020 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:32.010988    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-l224d_971ff8b7-349e-4224-b9c5-95798d96ab1d/discovery/0.log"
Jul 27 22:29:32.168637 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:32.168607    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-885jp_05798666-dd9a-440c-98c8-280baf18deb3/payload-processing/0.log"
Jul 27 22:29:32.186984 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:32.186962    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64884fdbff-j7wtj_4bb0a7c8-bc6b-449a-acbe-4ed3bf0c955f/router/0.log"
Jul 27 22:29:36.931592 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:36.931555    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c_2e1b4aee-0b22-46b0-80f7-362fd14d99fc/kube-storage-version-migrator-operator/1.log"
Jul 27 22:29:36.932502 ip-10-0-132-161 kubenswrapper[2578]: I0727 22:29:36.932480    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-ncc8c_2e1b4aee-0b22-46b0-80f7-362fd14d99fc/kube-storage-version-migrator-operator/0.log"